Skip to main content
Version: 0.1.11

Run Java Functions

Pulsar Functions is a succinct computing abstraction that Apache Pulsar enables users to express simple ETL and streaming tasks. Currently, Function Mesh supports using Java, Python, or Go programming language to define a YAML file of the Functions.

This document describes how to run Java Functions. To run a Java Functions in Function Mesh, you need to package the Functions and then submit it to a Pulsar cluster.

Package Java Functions​

After developing and testing your Pulsar Functions , you need to package it so that it can be submitted to a Pulsar cluster. You can package Java Functions to NAR/JAR packages or Docker images.

Java Functions packages​

This section describes how to package a Java Functions and upload it to the Pulsar package management service.

Build Java Functions packages​

This section describes how to build packages for Java Functions.

Prerequisites​
  • Apache Pulsar 2.8.0 or higher
  • Function Mesh v0.1.3 or higher
Steps​

To package a Functions in Java, follow these steps.

  1. Create a new Maven project with a pom.xml file. In the following code sample, the value of mainClass is your package name.

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>

        <groupId>java-function</groupId>
        <artifactId>java-function</artifactId>
        <version>1.0-SNAPSHOT</version>

        <dependencies>
            <dependency>
                <groupId>org.apache.pulsar</groupId>
                <artifactId>pulsar-functions-api</artifactId>
                <version>2.6.0</version>
            </dependency>
        </dependencies>

        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <configuration>
                        <appendAssemblyId>false</appendAssemblyId>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                        <archive>
                        <manifest>
                            <mainClass>org.example.test.ExclamationFunction</mainClass>
                        </manifest>
                    </archive>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id>
                            <phase>package</phase>
                            <goals>
                                <goal>assembly</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <configuration>
                        <source>8</source>
                        <target>8</target>
                    </configuration>
                </plugin>
            </plugins>
        </build>

    </project>

  2. Write a Java Functions.

    package org.example.test;

    import java.util.function.Function;

    public class ExclamationFunction implements Function<String, String> {
        @Override
        public String apply(String s) {
            return "This is my function!";
        }
    }

  3. Package the Java Functions.

    mvn package

    After the Java Functions is packaged, a target directory is created automatically. Open the target directory to check if there is a JAR package similar to java-function-1.0-SNAPSHOT.jar.

Upload Java Functions packages​

Use the pulsar-admin CLI tool to upload the package to the Pulsar package management service.

Note

Before uploading the package to Pulsar package management service, you need to enable the package management service in the broker.config file.

This example shows how to upload the package of the my-function@0.1 Functions to the Pulsar package management service.

bin/pulsar-admin packages upload function://my-tenant/my-ns/my-function@0.1 --path "/path/to/package-file" --description PACKAGE_DESCRIPTION

Then, you can define Functions CRDs by specifying the uploaded Functions package.

Docker images​

This section describes how to package a Pulsar Functions to a Docker image.

Prerequisites​

  • Apache Pulsar 2.7.0 or higher
  • Function Mesh v0.1.3 or higher

Build Docker images​

To build a Docker image, follow these steps.

  1. Package your Pulsar function. For details, see package Pulsar functions.

  2. Define a Dockerfile.

    This example shows how to define a Dockerfile with a JAR package (example-function.jar) of the Java Functions.

    # Use pulsar-functions-java-runner since we pack Java function
    FROM streamnative/pulsar-functions-java-runner:2.7.1
    # Copy function JAR package into /pulsar directory  
    COPY example-function.jar /pulsar/

Then, you can push the Functions Docker image into an image registry (such as the Docker Hub, or any private registry) and use the Functions Docker image to configure and submit the Functions to a Pulsar cluster.

Submit Java Functions​

After packaging your Pulsar Functions, you can submit your Pulsar Functions to a Pulsar cluster. This section describes how to submit a Java Functions through a Functions CRD. You can use the image field to specify the runner image use for creating the Java Functions. You can also specify the location where the package or the Docker image is stored.

  1. Define a Java Functions by using a YAML file and save the YAML file.

    • This example shows how to publish a java-function-sample Functions to a Pulsar cluster by using a JAR package called function://my-tenant/my-ns/my-function@0.1.

      apiVersion: compute.functionmesh.io/v1alpha1
      kind: Function
      metadata:
        name: java-function-sample
        namespace: default
      spec:
        image: streamnative/pulsar-functions-java-runner:2.7.1 # using java function runner
        className: exclamation_function.ExclamationFunction
        forwardSourceMessageProperty: true
        maxPendingAsyncRequests: 1000
        replicas: 1
        maxReplicas: 5
        logTopic: persistent://public/default/logging-function-logs
        input:
          topics:
          - persistent://public/default/java-function-input-topic
          typeClassName: java.lang.String
        output:
          topic: persistent://public/default/java-function-output-topic
          typeClassName: java.lang.String
        pulsar:
          pulsarConfig: "test-pulsar"
        java:
          extraDependenciesDir: random-dir/
          jar: my-function.jar # the package will download as this filename.
          jarLocation: function://my-tenant/my-ns/my-function@0.1 # function package URL

    • This example shows how to publish a java-function-sample Functions to a Pulsar cluster by using a Docker image.

      apiVersion: compute.functionmesh.io/v1alpha1
      kind: Function
      metadata:
        name: java-function-sample
        namespace: default
      spec:
        image: streamnative/example-function-image:latest # using function image here
        className: exclamation_function.ExclamationFunction
        forwardSourceMessageProperty: true
        maxPendingAsyncRequests: 1000
        replicas: 1
        maxReplicas: 5
        logTopic: persistent://public/default/logging-function-logs
        input:
          topics:
          - persistent://public/default/java-function-input-topic
          typeClassName: java.lang.String
        output:
          topic: persistent://public/default/java-function-output-topic
          typeClassName: java.lang.String
        pulsar:
          pulsarConfig: "test-pulsar"
        java:
          extraDependenciesDir: random-dir/
          jar: /pulsar/example-function.jar # the package location in image
          jarLocation: "" # leave empty since we will not download package from Pulsar Packages

  2. Apply the YAML file to create the Java Functions.

    kubectl apply -f /path/to/YAML/file

  3. Check whether the Java Functions is created successfully.

    kubectl get all