You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by si...@apache.org on 2019/06/18 07:47:35 UTC
[pulsar] branch master updated: [docs] Update
site2/docs/functions-quickstart.md (#4529)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2f53fb6 [docs] Update site2/docs/functions-quickstart.md (#4529)
2f53fb6 is described below
commit 2f53fb6f9350643e47f5abeb433d7fea72d0832b
Author: Jennifer Huang <47...@users.noreply.github.com>
AuthorDate: Tue Jun 18 15:47:29 2019 +0800
[docs] Update site2/docs/functions-quickstart.md (#4529)
### Motivation
While add "Write and run a Go function" section, improve doc on Pulsar Functions: quick start throughout.
### Modifications
1. Add a section of "Write and run a Go function".
2. Update the the first-person pronoun as the second-person pronoun.
3. Correct the usage of "Pulsar Functions" and "a Pulsar function". For the usage discussion, see #4507 .
4. Change the future tense into present tense.
5. Refine part of structure (heading level).
6. Refine sentences to be more clear and concise.
7. Refine some typos.
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? ( docs)
---
site2/docs/functions-quickstart.md | 181 ++++++++++++++++++++++++-------------
1 file changed, 117 insertions(+), 64 deletions(-)
diff --git a/site2/docs/functions-quickstart.md b/site2/docs/functions-quickstart.md
index 6b7a026..2e1265c 100644
--- a/site2/docs/functions-quickstart.md
+++ b/site2/docs/functions-quickstart.md
@@ -1,20 +1,20 @@
---
-id: functions-quickstart
-title: Get started with Pulsar Functions
-sidebar_label: Get started
+id: functions-quickstart
+title: Get started with Pulsar Functions
+sidebar_label: Get started
---
-This tutorial will walk you through running a [standalone](reference-terminology.md#standalone) Pulsar [cluster](reference-terminology.md#cluster) on your machine and then running your first Pulsar Functions using that cluster. The first function will run in local run mode (outside your Pulsar [cluster](reference-terminology.md#cluster)), while the second will run in cluster mode (inside your cluster).
+This tutorial walks you through running a [standalone](reference-terminology.md#standalone) Pulsar [cluster](reference-terminology.md#cluster) on your machine, and then running your first Pulsar Function using that cluster. The first Pulsar Function runs in local run mode (outside your Pulsar [cluster](reference-terminology.md#cluster)), while the second runs in cluster mode (inside your cluster).
-> In local run mode, your Pulsar Function will communicate with your Pulsar cluster but will run outside of the cluster.
+> In local run mode, Pulsar Functions communicate with Pulsar cluster, but run outside of the cluster.
## Prerequisites
-In order to follow along with this tutorial, you'll need to have [Maven](https://maven.apache.org/download.cgi) installed on your machine.
+Install [Maven](https://maven.apache.org/download.cgi) on your machine.
## Run a standalone Pulsar cluster
-In order to run our Pulsar Functions, we'll need to run a Pulsar cluster locally first. The easiest way to do that is to run Pulsar in [standalone](reference-terminology.md#standalone) mode. Follow these steps to start up a standalone cluster:
+In order to run Pulsar Functions, you need to run a Pulsar cluster locally first. The easiest way is to run Pulsar in [standalone](reference-terminology.md#standalone) mode. Follow these steps to start up a standalone cluster.
```bash
$ wget pulsar:binary_release_url
@@ -24,11 +24,11 @@ $ bin/pulsar standalone \
--advertised-address 127.0.0.1
```
-When running Pulsar in standalone mode, the `public` tenant and `default` namespace will be created automatically for you. That tenant and namespace will be used throughout this tutorial.
+When running Pulsar in standalone mode, the `public` tenant and the `default` namespace are created automatically. The tenant and namespace are used throughout this tutorial.
## Run a Pulsar Function in local run mode
-Let's start with a simple function that takes a string as input from a Pulsar topic, adds an exclamation point to the end of the string, and then publishes that new string to another Pulsar topic. Here's the code for the function:
+You can start with a simple function that takes a string as input from a Pulsar topic, adds an exclamation point to the end of the string, and then publishes the new string to another Pulsar topic. The following is the code for the function.
```java
package org.apache.pulsar.functions.api.examples;
@@ -43,7 +43,7 @@ public class ExclamationFunction implements Function<String, String> {
}
```
-A JAR file containing this and several other functions (written in Java) is included with the binary distribution you downloaded above (in the `examples` folder). To run the function in local mode, i.e. on our laptop but outside our Pulsar cluster:
+A JAR file containing this function and several other functions (written in Java) is included with the binary distribution you have downloaded (in the `examples` folder). Run the function in local mode on your laptop but outside your Pulsar cluster with the following commands.
```bash
$ bin/pulsar-admin functions localrun \
@@ -54,15 +54,15 @@ $ bin/pulsar-admin functions localrun \
--name exclamation
```
-> #### Multiple input topics allowed
+> #### Multiple input topics
>
-> In the example above, a single topic was specified using the `--inputs` flag. You can also specify multiple input topics as a comma-separated list using the same flag. Here's an example:
+> In the example above, a single topic is specified using the `--inputs` flag. You can also specify multiple input topics with a comma-separated list using the same flag.
>
> ```bash
> --inputs topic1,topic2
> ```
-We can open up another shell and use the [`pulsar-client`](reference-cli-tools.md#pulsar-client) tool to listen for messages on the output topic:
+You can open up another shell and use the [`pulsar-client`](reference-cli-tools.md#pulsar-client) tool to listen for messages on the output topic.
```bash
$ bin/pulsar-client consume persistent://public/default/exclamation-output \
@@ -70,9 +70,9 @@ $ bin/pulsar-client consume persistent://public/default/exclamation-output \
--num-messages 0
```
-> Setting the `--num-messages` flag to 0 means that the consumer will listen on the topic indefinitely (rather than only accepting a certain number of messages).
+> Setting the `--num-messages` flag to `0` means that consumers listen on the topic indefinitely, rather than only accepting a certain number of messages.
-With a listener up and running, we can open up another shell and produce a message on the input topic that we specified:
+With a listener up and running, you can open up another shell and produce a message on the input topic that you specify.
```bash
$ bin/pulsar-client produce persistent://public/default/exclamation-input \
@@ -80,26 +80,24 @@ $ bin/pulsar-client produce persistent://public/default/exclamation-input \
--messages "Hello world"
```
-In the output, you should see the following:
+When the message has been successfully processed by the exclamation function, you will see the following output. To shut down the function, press **Ctrl+C**.
```
----- got message -----
Hello world!
```
-Success! As you can see, the message has been successfully processed by the exclamation function. To shut down the function, simply hit **Ctrl+C**.
+### Process explanation
-Here's what happened:
-
-* The `Hello world` message that we published to the input topic (`persistent://public/default/exclamation-input`) was passed to the exclamation function that we ran on our machine
-* The exclamation function processed the message (providing a result of `Hello world!`) and published the result to the output topic (`persistent://public/default/exclamation-output`).
-* If our exclamation function *hadn't* been running, Pulsar would have durably stored the message data published to the input topic in [Apache BookKeeper](https://bookkeeper.apache.org) until a consumer consumed and acknowledged the message
+* The `Hello world` message you publish to the input topic (`persistent://public/default/exclamation-input`) is passed to the exclamation function.
+* The exclamation function processes the message (providing a result of `Hello world!`) and publishes the result to the output topic (`persistent://public/default/exclamation-output`).
+* If the exclamation function *does not* run, Pulsar will durably store the message data published to the input topic in [Apache BookKeeper](https://bookkeeper.apache.org) until a consumer consumes and acknowledges the message.
## Run a Pulsar Function in cluster mode
-[Local run mode](#run-a-pulsar-function-in-local-run-mode) is useful for development and experimentation, but if you want to use Pulsar Functions in a real Pulsar deployment, you'll want to run them in **cluster mode**. In this mode, Pulsar Functions run *inside* your Pulsar cluster and are managed using the same [`pulsar-admin functions`](reference-pulsar-admin.md#functions) interface that we've been using thus far.
+[Local run mode](#run-a-pulsar-function-in-local-run-mode) is useful for development and test. However, when you use Pulsar for real deployment, you run it in **cluster mode**. In cluster mode, Pulsar Functions run *inside* of your Pulsar cluster and are managed using the same [`pulsar-admin functions`](reference-pulsar-admin.md#functions) interface.
-This command, for example, would deploy the same exclamation function we ran locally above *in our Pulsar cluster* (rather than outside it):
+The following command deploys the same exclamation function you run locally in your Pulsar cluster, rather than outside of it.
```bash
$ bin/pulsar-admin functions create \
@@ -110,7 +108,7 @@ $ bin/pulsar-admin functions create \
--name exclamation
```
-You should see `Created successfully` in the output. Now, let's see a list of functions running in our cluster:
+You will see `Created successfully` in the output. Check the list of functions running in your cluster.
```bash
$ bin/pulsar-admin functions list \
@@ -118,7 +116,7 @@ $ bin/pulsar-admin functions list \
--namespace default
```
-We should see just the `exclamation` function listed there. We can also check the status of our deployed function using the `getstatus` command:
+You will see the `exclamation` function. Check the status of your deployed function using the `getstatus` command.
```bash
$ bin/pulsar-admin functions getstatus \
@@ -127,7 +125,7 @@ $ bin/pulsar-admin functions getstatus \
--name exclamation
```
-You should see this JSON output:
+You will see the following JSON output.
```json
{
@@ -140,7 +138,7 @@ You should see this JSON output:
}
```
-As we can see, (a) the instance is currently running and (b) there is one instance, with an ID of 0, running. We can get other information about the function (topics, tenant, namespace, etc.) using the `get` command instead of `getstatus`:
+As you can see, the instance is currently running, and an instance with the ID of `0` is running. With the `get` command, you can get other information about the function, for example, topics, tenant, namespace, and so on.
```bash
$ bin/pulsar-admin functions get \
@@ -149,7 +147,7 @@ $ bin/pulsar-admin functions get \
--name exclamation
```
-You should see this JSON output:
+You will see the following JSON output.
```json
{
@@ -166,7 +164,7 @@ You should see this JSON output:
}
```
-As we can see, the parallelism of the function is 1, meaning that only one instance of the function is running in our cluster. Let's update our function to a parallelism of 3 using the `update` command:
+As you can see, only one instance of the function is running in your cluster. Update the parallel functions to `3` using the `update` command.
```bash
$ bin/pulsar-admin functions update \
@@ -180,7 +178,7 @@ $ bin/pulsar-admin functions update \
--parallelism 3
```
-You should see `Updated successfully` in the output. If you run the `get` command from above for the function, you can see that the parallelism has increased to 3, meaning that there are now three instances of the function running in our cluster:
+You will see `Updated successfully` in the output. If you enter the `get` command, you see that the parallel functions are increased to `3`, meaning that three instances of the function are running in your cluster.
```json
{
@@ -197,7 +195,7 @@ You should see `Updated successfully` in the output. If you run the `get` comman
}
```
-Finally, we can shut down our running function using the `delete` command:
+Shut down the running function with the `delete` command.
```bash
$ bin/pulsar-admin functions delete \
@@ -206,31 +204,32 @@ $ bin/pulsar-admin functions delete \
--name exclamation
```
-If you see `Deleted successfully` in the output, then you've succesfully run, updated, and shut down a Pulsar Function running in cluster mode. Congrats! Now, let's go even further and run a brand new function in the next section.
+When you see `Deleted successfully` in the output, you've successfully run, updated, and shut down functions running in cluster mode.
-## Writing and running a new function
+## Write and run a new function
-> In order to write and run the [Python](functions-api.md#functions-for-python) function below, you'll need to install a few dependencies:
-> ```bash
-> $ pip install pulsar-client
-> ```
+In order to write and run [Python](functions-api.md#functions-for-python) functions, you need to install some dependencies.
+
+```bash
+$ pip install pulsar-client
+```
-In the above examples, we ran and managed a pre-written Pulsar Function and saw how it worked. To really get our hands dirty, let's write and our own function from scratch, using the Python API. This simple function will also take a string as input but it will reverse the string and publish the resulting, reversed string to the specified topic.
+In the examples above, you run and manage pre-written Pulsar Functions and learn how they work. You can also write your own functions with Python API. In the following example, the function takes a string as input, reverses the string, and publishes the reversed string to the specified topic.
-First, create a new Python file:
+First, create a new Python file.
```bash
$ touch reverse.py
```
-In that file, add the following:
+Add the following information in the Python file.
```python
def process(input):
return input[::-1]
```
-Here, the `process` method defines the processing logic of the Pulsar Function. It simply uses some Python slice magic to reverse each incoming string. Now, we can deploy the function using `create`:
+The `process` method defines the processing logic of Pulsar Functions. It uses Python slice magic to reverse each incoming string. You can deploy the function using the `create` command.
```bash
$ bin/pulsar-admin functions create \
@@ -243,7 +242,7 @@ $ bin/pulsar-admin functions create \
--name reverse
```
-If you see `Created successfully`, the function is ready to accept incoming messages. Because the function is running in cluster mode, we can **trigger** the function using the [`trigger`](reference-pulsar-admin.md#trigger) command. This command will send a message that we specify to the function and also give us the function's output. Here's an example:
+If you see `Created successfully`, the function is ready to accept incoming messages. Because the function is running in cluster mode, you can **trigger** the function using the [`trigger`](reference-pulsar-admin.md#trigger) command. This command sends a message that you specify to the function and returns the function output. The following is an example.
```bash
$ bin/pulsar-admin functions trigger \
@@ -253,40 +252,98 @@ $ bin/pulsar-admin functions trigger \
--trigger-value "sdrawrof won si tub sdrawkcab saw gnirts sihT"
```
-You should get this output:
+You will get the following output.
```
This string was backwards but is now forwards
```
-Once again, success! We created a brand new Pulsar Function, deployed it in our Pulsar standalone cluster in [cluster mode](#run-a-pulsar-function-in-cluster-mode) and successfully triggered the function. If you're ready for more, check out one of these docs:
+You have created a new Pulsar Function, deployed it in your Pulsar standalone cluster in [cluster mode](#run-a-pulsar-function-in-cluster-mode), and triggered the Function.
-## Packaging python dependencies
+## Write and run a Go function
+Go function depends on `pulsar-client-go`. Make sure that you have built `pulsar-client-go` before using Go function.
-For python functions requiring dependencies to be deployable in pulsar worker instances in an offline manner, we need to package the artifacts as below.
+To write and run a Go function, complete the following steps.
+1. Create a new Go file.
+
+```
+touch helloFunc.go
+```
-#### Client Requirements
-Following programs are required to be installed on the client machine
+2. Append a byte for messages from the input topic.
+The following is a `helloFunc.go` example. Each message from the input topic is appended with a `110` byte, and then delivered to the output topic.
```
-pip \\ rquired for getting python dependencies
+package main
+
+import (
+ "context"
+
+ "github.com/apache/pulsar/pulsar-function-go/pf"
+)
+
+func HandleResponse(ctx context.Context, in []byte) ([]byte, error) {
+ res := append(in, 110)
+ return res, nil
+}
+
+func main() {
+ pf.Start(HandleResponse)
+}
+```
+
+3. Compile code.
+
+```
+go build -o examplepulsar helloFunc.go
+```
+
+4. Run Go function.
+
+```
+$ bin/pulsar-admin functions create \
+ --go examplepulsar \
+ --inputs persistent://public/default/backwards \
+ --output persistent://public/default/forwards \
+ --tenant public \
+ --namespace default \
+ --name gofunc
+```
+
+If you see `Created successfully`, the function is ready to accept incoming messages. Start a producer and produce messages to the `backwards` input topic. Start a consumer and consume messages from the `forwards` output topic, you will see `110` is appended to all messages.
+
+The `--classname` parameter is not specified when running Go function, because there is no `Class` concept in Go, which is different from Java and Python.
+
+> Note
+> When you use the `--go` command to specify an executable file, make sure you have executable permissions.
+
+## Package Python dependencies
+
+When you deploy Python functions in a cluster offline, you need to package the required dependencies in a ZIP file before deployment.
+
+### Client requirements
+
+The following programs are required to be installed on the client machine.
+
+```
+pip \\ required for getting python dependencies
zip \\ for building zip archives
```
-#### Python Dependencies
+### Python dependencies
-A file named **requirements.txt** is needed with required dependencies for the python function
+A file named **requirements.txt** is needed with required dependencies for the Python function.
```
sh==1.12.14
```
-Prepare the pulsar function in folder called **src**.
+Prepare the Pulsar Function in the **src** folder.
-Run the following command to gather the python dependencies in the folder caller **deps**
+Run the following command to gather Python dependencies in the **deps** folder.
```
pip download \
@@ -298,7 +355,7 @@ pip download \
```
-Sample ouptut
+Sample output
```
Collecting sh==1.12.14 (from -r requirements.txt (line 1))
@@ -307,11 +364,11 @@ Collecting sh==1.12.14 (from -r requirements.txt (line 1))
Successfully downloaded sh
```
-**Note** pulsar-client is not needed as a dependency as it already installed in the worker node.
-
+> Note
+> `pulsar-client` is not needed as a dependency as it has already installed in the worker node.
-#### Packaging
-Create a destination folder with the desired pacaking name eg : **exclamation**, copy **src** and **deps** folder into it and finally compress the folder into a zip archive.
+#### Package
+Create a destination folder with the desired package name, for example, **exclamation**. Copy the **src** and **deps** folders into it, and compress the folder into a ZIP archive.
Sample sequence
@@ -329,8 +386,4 @@ drwxr-xr-x 3 a.ahmed staff 96 Nov 6 17:51 src
zip -r exclamation.zip exclamation
```
-Archive **exclamation.zip** can we deployed as function into a pulsar worker, the worker does not need internet connectivity to download packages as they are all included in the zip file.
-
-
-* [The Pulsar Functions API](functions-api.md)
-* [Deploying Pulsar Functions](functions-deploying.md)
+After package all the required dependencies into the **exclamation.zip** file, you can deploy functions in a Pulsar worker. The Pulsar worker does not need internet connectivity to download packages, because they are all included in the ZIP file.
\ No newline at end of file