Source CRD configurations
This document lists CRD configurations available for Pulsar source connectors. The source CRD configurations consist of source connector configurations and the common CRD configurations.
This table lists source configurations.
|The name of a source connector.|
|The class name of a source connector.|
|The tenant of a source connector.|
|The Pulsar cluster of a source connector.|
|The number of instances that you want to run this source connector. By default, the |
|The maximum number of Pulsar instances that you want to run for this source connector. When the value of the |
|The map to a ConfigMap specifying the configuration of a source connector.|
|The processing guarantees (delivery semantics) applied to the source connector. Available values: |
This section describes image options available for Pulsar source CRDs.
The base runner is an image base for other runners. The base runner is located at
./pulsar-functions-base-runner. The base runner image contains basic tool-chains like
/pulsar/lib to ensure that the
pulsar-admin CLI tool works properly to support Apache Pulsar Packages.
Function Mesh uses runner images as images of Pulsar connectors. Each runner image only contains necessary tool-chains and libraries for specified runtime.
Pulsar connectors supports using the Java runner images as their images. The Java runner is based on the base runner and contains the Java function instance to run Java functions or connectors. The
streamnative/pulsar-functions-java-runner Java runner is stored at the Docker Hub and is automatically updated to align with Apache Pulsar release.
The output topics of a Pulsar Function. This table lists options available for the
|The output topic of a Pulsar Function (If none is specified, no output is written).|
|The map of output topics to SerDe class names (as a JSON string).|
|The built-in schema type or custom schema class name to be used for messages sent by the function.|
|The producer specifications. Available options: |
|The map of output topics to Schema class names (as a JSON string).|
When you specify a function or connector, you can optionally specify how much of each resource they need. The resources available to specify are CPU and memory (RAM).
If the node where a Pod is running has enough of a resource available, it's possible (and allowed) for a Pod to use more resources than its
request for that resource. However, a Pod is not allowed to use more than its resource
In Function Mesh, the secret is defined through a
secretsMap. To use a secret, a Pod needs to reference the secret. Pods can consume secretsMaps as environment variables in a volume.
To use a secret as an environment variable in a Pod, follow these steps.
- Create a secret or use an existing one. Multiple Pods can reference the same secret.
- Modify your Pod definition in each container, which you want to consume the value of a secret key, to add an environment variable for each secret key that you want to consume.
- Modify your image and/or command line so that the program looks for values in the specified environment variables.
Function Mesh supports running Pulsar connectors in Java.
|Path to the JAR file for the connector.|
In Function Mesh, the Pulsar cluster is defined through a ConfigMap. Pods can consume ConfigMaps as environment variables in a volume. The Pulsar cluster ConfigMap defines the Pulsar cluster URLs.
|The Web service URL of the Pulsar cluster.|
|The broker service URL of the Pulsar cluster.|
Function Mesh supports customizing the Pod running connectors. This table lists sub-fields available for the
|Specify labels attached to a Pod.|
|Specify a map of key-value pairs. For a Pod running on a node, the node must have each of the indicated key-value pairs as labels.|
|Specify the scheduling constraints of a Pod.|
|Specify the tolerations of a Pod.|
|Specify the annotations attached to a Pod.|
|Specify the security context for a Pod.|
|It is the amount of time that Kubernetes gives for a Pod before terminating it.|
|It is a list of volumes that can be mounted by containers belonging to a Pod.|
|It is an optional list of references to secrets in the same namespace for pulling any of the images used by a Pod.|
|Initialization containers belonging to a Pod. A typical use case could be using an Initialization container to download a remote JAR to a local path.|
|Sidecar containers run together with the main function container in a Pod.|