Version: 0.1.9
Run Function Mesh
Function Mesh is a collection of functions collaborated together to process data with clearly-defined stages on Kubernetes environments.
This document describes how to run Function Mesh through a .yaml
file and submit Pulsar connectors and Pulsar functions through the Function Mesh CRD.
Prerequisites
- Create and connect to a Kubernetes cluster.
- Create a Pulsar cluster in the Kubernetes cluster.
- Install the Function Mesh Operator and CRD into Kubernetes cluster.
- package Pulsar Functions, see package Pulsar Functions.
- Package Pulsar connectors. For details, see package Pulsar connectors.
Steps
To create a Function Mesh, follow these steps.
Define a Function Mesh named
functionmesh-sample
by using a YAML file and save the YAML filefunctionmesh-sample.yaml
.apiVersion: compute.functionmesh.io/v1alpha1kind: FunctionMeshmetadata:name: functionmesh-samplespec:sinks:name: sink-sampleimage: streamnative/pulsar-io-elastic-search:2.7.1 # using connector image hereclassName: org.apache.pulsar.io.elasticsearch.ElasticSearchSinkreplicas: 1maxReplicas: 1input:topics:- persistent://public/default/inputtypeClassName: "[B"sinkConfig:elasticSearchUrl: "http://quickstart-es-http.default.svc.cluster.local:9200"indexName: "my_index"typeName: "doc"username: "elastic"password: "wJ757TmoXEd941kXm07Z2GW3"pulsar:pulsarConfig: "test-sink"resources:limits:cpu: "0.2"memory: 1.1Grequests:cpu: "0.1"memory: 1Gjava:extraDependenciesDir: random-dir/jar: connectors/pulsar-io-elastic-search-2.7.1.nar # the NAR location in imagejarLocation: "" # leave empty since we will not download package from Pulsar PackagesclusterName: test-pulsarautoAck: truefunctions:name: ex1className: exclamation_function.ExclamationFunctionreplicas: 1maxReplicas: 1logTopic: persistent://public/default/logging-function-loginput:topics:- persistent://public/default/functionmesh-input-topictypeClassName: java.lang.Stringoutput:topic: persistent://public/default/mid-topictypeClassName: java.lang.Stringpulsar:pulsarConfig: "mesh-test-pulsar"java:extraDependenciesDir: random-dir/jar: pulsar-functions-api-examples.jarjarLocation: public/default/nlu-test-functionmesh-ex1# following value must be provided if no auto-filling is enabledforwardSourceMessageProperty: trueautoAck: trueresources:requests:cpu: "0.1"memory: 1Glimits:cpu: "0.2"memory: 1.1GclusterName: test-pulsar---apiVersion: v1kind: ConfigMapmetadata:name: mesh-test-pulsardata:webServiceURL: http://test-pulsar-broker.default.svc.cluster.local:8080brokerServiceURL: pulsar://test-pulsar-broker.default.svc.cluster.local:6650Apply the YAML file to create the Function Mesh.
kubectl apply -f /path/to/functionmesh-sample.yamlCheck whether the Function Mesh is created successfully.
kubectl get all