You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/11/22 23:11:56 UTC
[GitHub] [pulsar] cbornet opened a new pull request, #18569: Add doc for builtin functions
cbornet opened a new pull request, #18569:
URL: https://github.com/apache/pulsar/pull/18569
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [x] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: <!-- ENTER URL HERE -->
<!--
After opening this PR, the build in apache/pulsar will fail and instructions will
be provided for opening a PR in the PR author's forked repository.
apache/pulsar pull requests should be first tested in your own fork since the
apache/pulsar CI based on GitHub Actions has constrained resources and quota.
GitHub Actions provides separate quota for pull requests that are executed in
a forked repository.
The tests will be run in the forked repository until all PR review comments have
been handled, the tests pass and the PR is approved by a reviewer.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
cbornet commented on PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#issuecomment-1325286048
I removed usage of `available-functions` and `reload` as they are not part of next/2.11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun merged pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
tisonkun merged PR #18569:
URL: https://github.com/apache/pulsar/pull/18569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
tisonkun commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1029984631
##########
site2/docs/functions-package-java.md:
##########
@@ -10,7 +10,23 @@ For the runtime Java version, refer to [Pulsar Runtime Java Version Recommendati
:::
-To package a Java function, complete the following steps.
+There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).
+
+:::note
+
+If you plan to package and distribute your function for others to use, you are obligated to
+license and copyright your own code properly. Remember to add the license and copyright to
+all libraries your code uses and to your distribution.
+
+:::
+
+> If you use the [NAR](#package-as-nar) method, the NAR plugin
+automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
+licensing and copyrights of all libraries of your function.
Review Comment:
They seem similar notes/tips, why different syntax?
You may read the syntax guide (https://pulsar.apache.org/contribute/document-syntax/#admonitions) for what is expected here.
##########
site2/docs/functions-package-java.md:
##########
@@ -106,3 +122,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
+
+# Package as NAR
+
+To package a Java function as NAR, complete the following steps.
+
+1. Create a new maven project with a pom file.
+
+ ```xml
+ <?xml version="1.0" encoding="UTF-8"?>
+ <project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <groupId>java-function</groupId>
+ <artifactId>java-function</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar-functions-api</artifactId>
+ <version>2.10.0</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-nar-maven-plugin</artifactId>
+ <version>1.2.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <release>17</release>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ </project>
+ ```
+
+You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.
+
+```yaml
+name: java-function
+description: my java function
+functionClass: org.example.test.ExclamationFunction
+```
Review Comment:
Also one-level ident?
##########
site2/docs/functions-deploy-cluster-builtin.md:
##########
@@ -0,0 +1,30 @@
+---
+id: functions-deploy-cluster-builtin
+title: Built-in functions
+sidebar_label: "Built-in functions"
+---
+
+Similar to built-in connectors, the code of Java functions [packaged as NAR](functions-package-java.md) that are placed in the `functions` directory of the function worker are loaded at startup and can be referenced when creating a function.
+
+For instance if you have a built-in function with name `exclamation` in its `pulsar-io.yaml`, you can create a function instance with:
+
+```bash
+bin/pulsar-admin functions create \
+ --function-type exclamation \
+ --inputs persistent://public/default/input-1 \
+ --output persistent://public/default/output-1
+```
+
+To get the list of available built-in Functions, use the `available-functions` command:
+
+```bash
+bin/pulsar-admin functions available-functions
+```
+
+If you add or delete a nar file in a `functions` folder, reload the available built-in functions before using it.
+
+```bash
+bin/pulsar-admin functions reload
+```
+
+
Review Comment:
```suggestion
```
Trim whitespace.
##########
site2/docs/functions-package-java.md:
##########
@@ -106,3 +122,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
+
+# Package as NAR
+
+To package a Java function as NAR, complete the following steps.
+
+1. Create a new maven project with a pom file.
+
+ ```xml
+ <?xml version="1.0" encoding="UTF-8"?>
+ <project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <groupId>java-function</groupId>
+ <artifactId>java-function</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar-functions-api</artifactId>
+ <version>2.10.0</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-nar-maven-plugin</artifactId>
+ <version>1.2.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <release>17</release>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ </project>
+ ```
+
+You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.
+
+```yaml
+name: java-function
+description: my java function
+functionClass: org.example.test.ExclamationFunction
+```
+
+2. Package your Java function.
+
+ ```bash
+ mvn package
Review Comment:
```suggestion
mvn package
```
Remove leading whitespace. Ditto others.
##########
site2/docs/functions-package-java.md:
##########
@@ -106,3 +122,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
+
+# Package as NAR
+
+To package a Java function as NAR, complete the following steps.
+
+1. Create a new maven project with a pom file.
+
+ ```xml
+ <?xml version="1.0" encoding="UTF-8"?>
+ <project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <groupId>java-function</groupId>
+ <artifactId>java-function</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar-functions-api</artifactId>
+ <version>2.10.0</version>
Review Comment:
```suggestion
<version>@pulsar:version@</version>
```
Perhaps resolve to the lastest version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1031358471
##########
site2/docs/functions-package-java.md:
##########
@@ -4,13 +4,33 @@ title: Package Java Functions
sidebar_label: "Package Java Functions"
---
+There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).
+
+:::note
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1030249585
##########
site2/docs/functions-package-java.md:
##########
@@ -106,3 +122,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
+
+# Package as NAR
+
+To package a Java function as NAR, complete the following steps.
+
+1. Create a new maven project with a pom file.
+
+ ```xml
+ <?xml version="1.0" encoding="UTF-8"?>
+ <project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <groupId>java-function</groupId>
+ <artifactId>java-function</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar-functions-api</artifactId>
+ <version>2.10.0</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-nar-maven-plugin</artifactId>
+ <version>1.2.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <release>17</release>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ </project>
+ ```
+
+You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.
+
+```yaml
+name: java-function
+description: my java function
+functionClass: org.example.test.ExclamationFunction
+```
Review Comment:
done
##########
site2/docs/functions-package-java.md:
##########
@@ -10,7 +10,23 @@ For the runtime Java version, refer to [Pulsar Runtime Java Version Recommendati
:::
-To package a Java function, complete the following steps.
+There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).
+
+:::note
+
+If you plan to package and distribute your function for others to use, you are obligated to
+license and copyright your own code properly. Remember to add the license and copyright to
+all libraries your code uses and to your distribution.
+
+:::
+
+> If you use the [NAR](#package-as-nar) method, the NAR plugin
+automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
+licensing and copyrights of all libraries of your function.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#issuecomment-1326556459
Meriging...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1031017896
##########
site2/docs/functions-package-java.md:
##########
@@ -4,13 +4,33 @@ title: Package Java Functions
sidebar_label: "Package Java Functions"
---
+There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).
+
+:::note
Review Comment:
With the note and tip newly added, there will be three consecutive color blocks (but in different colors), which is a bit overwhelming in visuals IMO. Does it help by combining the three items in a single note/tip block?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1031021693
##########
site2/docs/functions-deploy-cluster-builtin.md:
##########
@@ -0,0 +1,16 @@
+---
+id: functions-deploy-cluster-builtin
+title: Built-in functions
+sidebar_label: "Built-in functions"
Review Comment:
It's a task-oriented topic for users. So it makes more sense to use an imperative title, like `Use built-in functions`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1031018443
##########
site2/docs/functions-deploy-cluster-builtin.md:
##########
@@ -0,0 +1,16 @@
+---
+id: functions-deploy-cluster-builtin
Review Comment:
Suggest adding the link of this new topic to the [parent topic](https://pulsar.apache.org/docs/next/functions-deploy-cluster) to update the map for users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18569: [improve][doc] Add doc for builtin functions
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18569:
URL: https://github.com/apache/pulsar/pull/18569#discussion_r1031358948
##########
site2/docs/functions-deploy-cluster-builtin.md:
##########
@@ -0,0 +1,16 @@
+---
+id: functions-deploy-cluster-builtin
Review Comment:
done
##########
site2/docs/functions-deploy-cluster-builtin.md:
##########
@@ -0,0 +1,16 @@
+---
+id: functions-deploy-cluster-builtin
+title: Built-in functions
+sidebar_label: "Built-in functions"
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org