[GitHub] [pulsar] tuteng commented on a change in pull request #5071: [Doc] Add *How to Use Pulsar Connectors* Guide

[GitBox <gi...@apache.org>]

tuteng commented on a change in pull request #5071: [Doc] Add *How to Use Pulsar Connectors* Guide
URL: https://github.com/apache/pulsar/pull/5071#discussion_r319419034
 
 

 ##########
 File path: site2/docs/io-use.md
 ##########
 @@ -0,0 +1,1411 @@
+---
+id: io-use
+title: How to use Pulsar connectors
+sidebar_label: Use
+---
+
+This guide describes how to use Pulsar connectors.
+
+## Install a connector
+
+Pulsar bundles several [builtin connectors](io-connectors.md) used to move data in and out of commonly used systems (such as database and messaging system). Optionally, you can create and use your desired non-builtin connectors.
+
+> #### Note
+> 
+> When using a non-builtin connector, you need to specify the path of a archive file for the connector.
+
+To set up a builtin connector, follow
+the instructions [here](getting-started-standalone.md#installing-builtin-connectors).
+
+After the setup, the builtin connector is automatically discovered by Pulsar brokers (or function-workers), so no additional installation steps are required.
+
+## Configure a connector
+
+You can configure the following information:
+
+* [Configure a default storage location for a connector](#configure-a-default-storage-location-for-a-connector)
+
+* [Configure a connector with a YAML file](#configure-a-connector-with-yaml-file)
+
+### Configure a default storage location for a connector
+
+To configure a default folder for builtin connectors, set the `connectorsDirectory` parameter in the `./conf/functions_worker.yml` configuration file.
+
+**Example**
+
+Set the `./connectors` folder as the default storage location for builtin connectors.
+
+```
+########################
+# Connectors
+########################
+
+connectorsDirectory: ./connectors
+```
+
+### Configure a connector with a YAML file
+
+To configure a connector, you need to provide a YAML configuration file when creating a connector.
+
+The YAML configuration file tells Pulsar where to locate connectors and how to connect connectors with Pulsar topics.
+
+**Example 1**
+
+Below is a YAML configuration file of a Cassandra sink, which tells Pulsar:
+
+* Which Cassandra cluster to connect
+  
+* What is the `keyspace` and `columnFamily` to be used in Cassandra for collecting data
+  
+* How to map Pulsar messages into Cassandra table key and columns
+
+```shell
+tenant: public
+namespace: default
+name: cassandra-test-sink
+...
+# cassandra specific config
+configs:
+    roots: "localhost:9042"
+    keyspace: "pulsar_test_keyspace"
+    columnFamily: "pulsar_test_table"
+    keyname: "key"
+    columnName: "col"
+```
+
+**Example 2**
+
+Below is a YAML configuration file of a Kafka source.
+
+```shell
+configs:
+   bootstrapServers: "pulsar-kafka:9092"
+   groupId: "test-pulsar-io"
+   topic: "my-topic"
+   sessionTimeoutMs: "10000"
+   autoCommitEnabled: "false"
+```
+
+**Example 3**
+
+Below is a YAML configuration file of a MySQL JDBC sink.
+
+```shell
+configs:
+   userName: "root"
+   password: "jdbc"
+   jdbcUrl: "jdbc:mysql://127.0.0.1:3306/test_jdbc"
+   tableName: "test_jdbc"
+```
+
+## Get available connectors
+
+Before starting using connectors, you can perform the following operations:
+
+* [Reload connectors](#reload)
+  
+* [Get a list of available connectors](#get-available-connectors)
+
+### `reload`
+
+If you add or delete a nar file in a connector folder, reload the available builtin connector before using it.
+
+#### Source
+
+Use the `reload` subcommand.
+
+```shell
+$ pulsar-admin sources reload
+```
+
+For more information, see [`here`](reference-connector-admin/#reload).
+
+#### Sink
+
+Use the `reload` subcommand.
+
+```shell
+$ pulsar-admin sinks reload
+```
+
+For more information, see [`here`](reference-connector-admin/#reload-1).
+
+### `available`
+
+After reloading connectors (optional), you can get a list of available connectors.
+
+#### Source
+
+Use the `available-sources` subcommand.
+
+```shell
+$ pulsar-admin sources available-sources
+```
+
+#### Sink
+
+Use the `available-sinks` subcommand.
+
+```shell
+$ pulsar-admin sinks available-sinks
+```
+
+## Run a connector
+
+To run a connector, you can perform the following operations:
+
+* [Create a connector](#create)
+  
+* [Start a connector](#start)
+
+* [Run a connector locally](#localrun)
+
+### `create`
+
+You can create a connector using **Admin CLI**, **REST API** or **JAVA admin API**.
+
+#### Source
+
+Create a source connector.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--Admin CLI-->
+
+Use the `create` subcommand.
+
+```
+$ pulsar-admin sources create options
+```
+
+For more information, see [here](reference-connector-admin.md#create).
+
+<!--REST API-->
+
+Send a `POST` request to this endpoint: {@inject: endpoint|POST|/admin/v3/sources/:tenant/:namespace/:sourceName|operation/registerSource}
+
+<!--Java Admin API-->
+
+* Create a source connector with a **local file**.
+
+    ```java
+    void createSource(SourceConfig sourceConfig,
+                      String fileName)
+               throws PulsarAdminException
+    ```
+
+    **Parameter**
+
+    |Name|Description
+    |---|---
+    `sourceConfig` | The source configuration object
+
+   **Exception**
+
+    |Name|Description|
+    |---|---
+    | `PulsarAdminException` | Unexpected error
+
+    For more information, see [`createSource`](https://pulsar.apache.org/api/admin/org/apache/pulsar/client/admin/Source.html#createSource-SourceConfig-java.lang.String-).
+
+* Create a source connector using a **remote file** with a URL from which fun-pkg can be downloaded. 
+
+    ```java
+    void createSourceWithUrl(SourceConfig sourceConfig,
+                             String pkgUrl)
+                      throws PulsarAdminException
+    ```
+
+    Supported URLs are `http` and `file`.
+
+    **Example**
+
+    * HTTP: http://www.repo.com/fileName.jar
+
+    * File: file:/dir/fileName.jar
 
 Review comment:
   File protocol format: file:///dir/fileName.jar

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services