You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by mm...@apache.org on 2019/05/28 20:51:56 UTC
[pulsar] branch master updated: update sinks and sources
documentation (#4388)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 330b75e update sinks and sources documentation (#4388)
330b75e is described below
commit 330b75e63ff1523ec06ef47083f654812b10e0a5
Author: Boyang Jerry Peng <je...@gmail.com>
AuthorDate: Tue May 28 13:51:48 2019 -0700
update sinks and sources documentation (#4388)
---
site/docs/latest/io/quickstart.md | 12 ++++----
site2/docs/io-cdc-canal.md | 2 +-
site2/docs/io-cdc-debezium.md | 2 +-
site2/docs/io-managing.md | 20 ++++++-------
site2/docs/io-quickstart.md | 42 +++++++++++++--------------
site2/docs/reference-pulsar-admin.md | 56 ++++++++++++++++++------------------
site2/docs/sql-getting-started.md | 2 +-
7 files changed, 68 insertions(+), 68 deletions(-)
diff --git a/site/docs/latest/io/quickstart.md b/site/docs/latest/io/quickstart.md
index f394d0c..e3f340c 100644
--- a/site/docs/latest/io/quickstart.md
+++ b/site/docs/latest/io/quickstart.md
@@ -41,19 +41,19 @@ You can use the [`create`](../../reference/CliTools#pulsar-admin-source-create)
You can submit a sink to be run in an existing Pulsar cluster using a command of this form:
```bash
-$ ./bin/pulsar-admin sink create --className <classname> --jar <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
+$ ./bin/pulsar-admin sinks create --className <classname> --jar <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
```
Here’s an example command:
```bash
-bin/pulsar-admin source create --className org.apache.pulsar.io.twitter.TwitterFireHose --jar ~/application.jar --tenant test --namespace ns1 --name twitter-source --destinationTopicName twitter_data
+bin/pulsar-admin sources create --className org.apache.pulsar.io.twitter.TwitterFireHose --jar ~/application.jar --tenant test --namespace ns1 --name twitter-source --destinationTopicName twitter_data
```
Instead of submitting a source to run on an existing Pulsar cluster, you alternatively can run a source as a process on your local machine:
```bash
-bin/pulsar-admin source localrun --className org.apache.pulsar.io.twitter.TwitterFireHose --jar ~/application.jar --tenant test --namespace ns1 --name twitter-source --destinationTopicName twitter_data
+bin/pulsar-admin sources localrun --className org.apache.pulsar.io.twitter.TwitterFireHose --jar ~/application.jar --tenant test --namespace ns1 --name twitter-source --destinationTopicName twitter_data
```
### Running Sinks
@@ -61,19 +61,19 @@ bin/pulsar-admin source localrun --className org.apache.pulsar.io.twitter.Twitt
You can submit a sink to be run in an existing Pulsar cluster using a command of this form:
```bash
-./bin/pulsar-admin sink create --className <classname> --jar <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
+./bin/pulsar-admin sinks create --className <classname> --jar <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
```
Here’s an example command:
```bash
-./bin/pulsar-admin sink create --className org.apache.pulsar.io.cassandra --jar ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
+./bin/pulsar-admin sinks create --className org.apache.pulsar.io.cassandra --jar ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
```
Instead of submitting a sink to run on an existing Pulsar cluster, you alternatively can run a sink as a process on your local machine:
```bash
-./bin/pulsar-admin sink localrun --className org.apache.pulsar.io.cassandra --jar ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
+./bin/pulsar-admin sinks localrun --className org.apache.pulsar.io.cassandra --jar ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
```
## Available connectors
diff --git a/site2/docs/io-cdc-canal.md b/site2/docs/io-cdc-canal.md
index f388c1e..e5d53aa 100644
--- a/site2/docs/io-cdc-canal.md
+++ b/site2/docs/io-cdc-canal.md
@@ -144,7 +144,7 @@ docker cp pulsar-client.py pulsar-standalone:/pulsar/
```$bash
docker exec -it pulsar-standalone /bin/bash
wget http://apache.01link.hk/pulsar/pulsar-2.3.0/connectors/pulsar-io-canal-2.3.0.nar -P connectors
-./bin/pulsar-admin source localrun --archive ./connectors/pulsar-io-canal-2.3.0.nar --classname org.apache.pulsar.io.canal.CanalStringSource --tenant public --namespace default --name canal --destination-topic-name my-topic --source-config-file /pulsar/conf/canal-mysql-source-config.yaml --parallelism 1
+./bin/pulsar-admin sources localrun --archive ./connectors/pulsar-io-canal-2.3.0.nar --classname org.apache.pulsar.io.canal.CanalStringSource --tenant public --namespace default --name canal --destination-topic-name my-topic --source-config-file /pulsar/conf/canal-mysql-source-config.yaml --parallelism 1
```
- Consumption data
diff --git a/site2/docs/io-cdc-debezium.md b/site2/docs/io-cdc-debezium.md
index d5c1e5e..413d3f9 100644
--- a/site2/docs/io-cdc-debezium.md
+++ b/site2/docs/io-cdc-debezium.md
@@ -112,7 +112,7 @@ Here is a simple example to store MySQL change data using above example config.
- Start pulsar debezium connector, with local run mode, and using above yaml config file. Please make sure that the nar file is available as configured in path `connectors/pulsar-io-debezium-mysql-{{pulsar:version}}.nar`.
```$bash
- bin/pulsar-admin source localrun --sourceConfigFile debezium-mysql-source-config.yaml
+ bin/pulsar-admin sources localrun --sourceConfigFile debezium-mysql-source-config.yaml
```
- Subscribe the topic for table `inventory.products`.
diff --git a/site2/docs/io-managing.md b/site2/docs/io-managing.md
index 72210d1..2f29a5a 100644
--- a/site2/docs/io-managing.md
+++ b/site2/docs/io-managing.md
@@ -54,19 +54,19 @@ Pulsar connectors can be managed using the [`source`](reference-pulsar-admin.md#
You can submit a source to be run in an existing Pulsar cluster using a command of this form:
```bash
-$ ./bin/pulsar-admin source create --classname <classname> --archive <jar-location> --tenant <tenant> --namespace <namespace> --name <source-name> --destination-topic-name <output-topic>
+$ ./bin/pulsar-admin sources create --classname <classname> --archive <jar-location> --tenant <tenant> --namespace <namespace> --name <source-name> --destination-topic-name <output-topic>
```
Here’s an example command:
```bash
-bin/pulsar-admin source create --classname org.apache.pulsar.io.twitter.TwitterFireHose --archive ~/application.jar --tenant test --namespace ns1 --name twitter-source --destination-topic-name twitter_data
+bin/pulsar-admin sources create --classname org.apache.pulsar.io.twitter.TwitterFireHose --archive ~/application.jar --tenant test --namespace ns1 --name twitter-source --destination-topic-name twitter_data
```
Instead of submitting a source to run on an existing Pulsar cluster, you alternatively can run a source as a process on your local machine:
```bash
-bin/pulsar-admin source localrun --classname org.apache.pulsar.io.twitter.TwitterFireHose --archive ~/application.jar --tenant test --namespace ns1 --name twitter-source --destination-topic-name twitter_data
+bin/pulsar-admin sources localrun --classname org.apache.pulsar.io.twitter.TwitterFireHose --archive ~/application.jar --tenant test --namespace ns1 --name twitter-source --destination-topic-name twitter_data
```
If you are submitting a built-in source, you don't need to specify `--classname` and `--archive`.
@@ -74,7 +74,7 @@ You can simply specify the source type `--source-type`. The command to submit a
in following form:
```bash
-./bin/pulsar-admin source create \
+./bin/pulsar-admin sources create \
--tenant <tenant> \
--namespace <namespace> \
--name <source-name> \
@@ -85,7 +85,7 @@ in following form:
Here's an example to submit a Kafka source:
```bash
-./bin/pulsar-admin source create \
+./bin/pulsar-admin sources create \
--tenant test-tenant \
--namespace test-namespace \
--name test-kafka-source \
@@ -98,19 +98,19 @@ Here's an example to submit a Kafka source:
You can submit a sink to be run in an existing Pulsar cluster using a command of this form:
```bash
-./bin/pulsar-admin sink create --classname <classname> --archive <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
+./bin/pulsar-admin sinks create --classname <classname> --archive <jar-location> --tenant test --namespace <namespace> --name <sink-name> --inputs <input-topics>
```
Here’s an example command:
```bash
-./bin/pulsar-admin sink create --classname org.apache.pulsar.io.cassandra --archive ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
+./bin/pulsar-admin sinks create --classname org.apache.pulsar.io.cassandra --archive ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
```
Instead of submitting a sink to run on an existing Pulsar cluster, you alternatively can run a sink as a process on your local machine:
```bash
-./bin/pulsar-admin sink localrun --classname org.apache.pulsar.io.cassandra --archive ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
+./bin/pulsar-admin sinks localrun --classname org.apache.pulsar.io.cassandra --archive ~/application.jar --tenant test --namespace ns1 --name cassandra-sink --inputs test_topic
```
If you are submitting a built-in sink, you don't need to specify `--classname` and `--archive`.
@@ -118,7 +118,7 @@ You can simply specify the sink type `--sink-type`. The command to submit a buil
in following form:
```bash
-./bin/pulsar-admin sink create \
+./bin/pulsar-admin sinks create \
--tenant <tenant> \
--namespace <namespace> \
--name <sink-name> \
@@ -129,7 +129,7 @@ in following form:
Here's an example to submit a Cassandra sink:
```bash
-./bin/pulsar-admin sink create \
+./bin/pulsar-admin sinks create \
--tenant test-tenant \
--namespace test-namespace \
--name test-cassandra-sink \
diff --git a/site2/docs/io-quickstart.md b/site2/docs/io-quickstart.md
index 8d94183..e58b623 100644
--- a/site2/docs/io-quickstart.md
+++ b/site2/docs/io-quickstart.md
@@ -180,7 +180,7 @@ Pulsar provides the [CLI](reference-cli-tools.md) for running and managing Pulsa
We can run following command to sink a sink connector with type `cassandra` and config file `examples/cassandra-sink.yml`.
```shell
-bin/pulsar-admin sink create \
+bin/pulsar-admin sinks create \
--tenant public \
--namespace default \
--name cassandra-test-sink \
@@ -200,7 +200,7 @@ for inspecting and managing the IO connectors.
#### Retrieve Sink Info
```bash
-bin/pulsar-admin sink get \
+bin/pulsar-admin sinks get \
--tenant public \
--namespace default \
--name cassandra-test-sink
@@ -237,7 +237,7 @@ Example output:
#### Check Sink running status
```bash
-bin/pulsar-admin sink status \
+bin/pulsar-admin sinks status \
--tenant public \
--namespace default \
--name cassandra-test-sink
@@ -279,7 +279,7 @@ for i in {0..9}; do bin/pulsar-client produce -m "key-$i" -n 1 test_cassandra; d
Inspect the sink running status again. You should be able to see 10 messages are processed by the Cassandra sink.
```bash
-bin/pulsar-admin sink status \
+bin/pulsar-admin sinks status \
--tenant public \
--namespace default \
--name cassandra-test-sink
@@ -339,7 +339,7 @@ cqlsh:pulsar_test_keyspace> select * from pulsar_test_table;
### Delete the Cassandra Sink
```shell
-bin/pulsar-admin sink delete \
+bin/pulsar-admin sinks delete \
--tenant public \
--namespace default \
--name cassandra-test-sink
@@ -495,7 +495,7 @@ Pulsar provides the [CLI](admin-api-overview.md) for running and managing Pulsar
This example creates a sink connector and specifies the desired information.
```text
-$ bin/pulsar-admin sink create \
+$ bin/pulsar-admin sinks create \
--archive ./connectors/pulsar-io-jdbc-{{pulsar:version}}.nar \
--inputs pulsar-mysql-jdbc-sink-topic \
--name pulsar-mysql-jdbc-sink \
@@ -514,7 +514,7 @@ Once the command is executed, Pulsar will create a sink connector named _pulsar-
> `--name` | The name of the sink. | _pulsar-mysql-jdbc-sink_
> `--sink-config-file` | The path to a YAML config file specifying the configuration of the sink. | _pulsar-mysql-jdbc-sink.yaml_
> `--parallelism` | The parallelism factor of the sink. <br> For example, the number of sink instances to run. | _1_
-> For more information about `pulsar-admin sink create options`, see [here](pulsar-admin/#create-3).
+> For more information about `pulsar-admin sinks create options`, see [here](pulsar-admin/#create-3).
The sink has been created successfully if the following message appears.
@@ -529,7 +529,7 @@ The sink has been created successfully if the following message appears.
This example lists all running sink connectors.
```text
-$ bin/pulsar-admin sink list \
+$ bin/pulsar-admin sinks list \
--tenant public \
--namespace default
```
@@ -547,7 +547,7 @@ The result shows that only the _mysql-jdbc-sink_ sink is running.
This example gets the information about the _pulsar-mysql-jdbc-sink_ sink connector.
```text
-$ bin/pulsar-admin sink get \
+$ bin/pulsar-admin sinks get \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink
@@ -584,7 +584,7 @@ The result show the information of the sink connector, including tenant, namespa
This example checks the current status of the _pulsar-mysql-jdbc-sink_ sink connector.
```text
-$ bin/pulsar-admin sink status \
+$ bin/pulsar-admin sinks status \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink
@@ -620,7 +620,7 @@ The result shows the current status of sink connector, including the number of i
This example stops the _pulsar-mysql-jdbc-sink_ sink instance.
```text
-$ bin/pulsar-admin sink stop \
+$ bin/pulsar-admin sinks stop \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink \
@@ -638,7 +638,7 @@ The sink instance has been stopped successfully if the following message disappe
This example starts the _pulsar-mysql-jdbc-sink_ sink instance.
```text
-$ bin/pulsar-admin sink start \
+$ bin/pulsar-admin sinks start \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink \
@@ -652,24 +652,24 @@ The sink instance has been started successfully if the following message disappe
```
> #### Tip
-> Optionally, you can run a standalone sink connector using `pulsar-admin sink localrun options`.
+> Optionally, you can run a standalone sink connector using `pulsar-admin sinks localrun options`.
>
-> Note that `pulsar-admin sink localrun options` runs a sink connector locally, while `pulsar-admin sink start options` starts a sink connector in a cluster.
+> Note that `pulsar-admin sinks localrun options` runs a sink connector locally, while `pulsar-admin sinks start options` starts a sink connector in a cluster.
>
-> For more information about `pulsar-admin sink localrun options`, see [here](pulsar-admin/#localrun-1).
+> For more information about `pulsar-admin sinks localrun options`, see [here](pulsar-admin/#localrun-1).
### Update a JDBC sink
This example updates the parallelism of the _pulsar-mysql-jdbc-sink_ sink connector to 2.
```text
-$ bin/pulsar-admin sink update \
+$ bin/pulsar-admin sinks update \
--name pulsar-mysql-jdbc-sink \
--parallelism 2
```
> #### Tip
-> For more information about `pulsar-admin sink update options`, see [here](pulsar-admin/#update-2).
+> For more information about `pulsar-admin sinks update options`, see [here](pulsar-admin/#update-2).
The sink connector has been updated successfully if the following message disappears.
@@ -680,7 +680,7 @@ The sink connector has been updated successfully if the following message disapp
This example double-checks the information.
```text
-$ bin/pulsar-admin sink get \
+$ bin/pulsar-admin sinks get \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink
@@ -717,14 +717,14 @@ The result shows that the parallelism is 2.
This example deletes the _pulsar-mysql-jdbc-sink_ sink connector.
```text
-$ bin/pulsar-admin sink delete \
+$ bin/pulsar-admin sinks delete \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink
```
> #### Tip
-> For more information about `pulsar-admin sink delete options`, see [here](pulsar-admin/#delete-4).
+> For more information about `pulsar-admin sinks delete options`, see [here](pulsar-admin/#delete-4).
The sink connector has been deleted successfully if the following message appears.
@@ -735,7 +735,7 @@ The sink connector has been deleted successfully if the following message appear
This example double-checks the existence of the sink connector.
```text
-$ bin/pulsar-admin sink get \
+$ bin/pulsar-admin sinks get \
--tenant public \
--namespace default \
--name pulsar-mysql-jdbc-sink
diff --git a/site2/docs/reference-pulsar-admin.md b/site2/docs/reference-pulsar-admin.md
index c57c08b..e0cae4c 100644
--- a/site2/docs/reference-pulsar-admin.md
+++ b/site2/docs/reference-pulsar-admin.md
@@ -18,8 +18,8 @@ Commands
* `functions`
* `namespaces`
* `ns-isolation-policy`
-* `sink`
-* `source`
+* `sinks`
+* `sources`
* `topics`
* `tenants`
* `resource-quotas`
@@ -1467,13 +1467,13 @@ Options
|`--broker`|Broker name to get namespace-isolation policies attached to it||
-## `sink`
+## `sinks`
An interface for managing Pulsar IO sinks (egress data from Pulsar)
Usage
```bash
-$ pulsar-admin sink subcommand
+$ pulsar-admin sinks subcommand
```
Subcommands
@@ -1495,7 +1495,7 @@ Submit a Pulsar IO sink connector to run in a Pulsar cluster
Usage
```bash
-$ pulsar-admin sink create options
+$ pulsar-admin sinks create options
```
Options
@@ -1529,7 +1529,7 @@ Update a Pulsar IO sink connector
Usage
```bash
-$ pulsar-admin sink update options
+$ pulsar-admin sinks update options
```
Options
@@ -1563,7 +1563,7 @@ Stops a Pulsar IO sink connector
Usage
```bash
-$ pulsar-admin sink delete options
+$ pulsar-admin sinks delete options
```
Options
@@ -1579,7 +1579,7 @@ List all running Pulsar IO sink connectors
Usage
```bash
-$ pulsar-admin sink list options
+$ pulsar-admin sinks list options
```
Options
@@ -1594,7 +1594,7 @@ Gets the information about a Pulsar IO sink connector
Usage
```bash
-$ pulsar-admin sink get options
+$ pulsar-admin sinks get options
```
Options
@@ -1610,7 +1610,7 @@ Check the current status of a Pulsar Sink
Usage
```bash
-$ pulsar-admin sink status options
+$ pulsar-admin sinks status options
```
Options
@@ -1627,7 +1627,7 @@ Stops sink instance
Usage
```bash
-$ pulsar-admin sink stop options
+$ pulsar-admin sinks stop options
```
Options
@@ -1644,7 +1644,7 @@ Starts sink instance
Usage
```bash
-$ pulsar-admin sink start options
+$ pulsar-admin sinks start options
```
Options
@@ -1661,7 +1661,7 @@ Restart sink instance
Usage
```bash
-$ pulsar-admin sink restart options
+$ pulsar-admin sinks restart options
```
Options
@@ -1678,7 +1678,7 @@ Run a Pulsar IO sink connector locally (rather than deploying it to the Pulsar c
Usage
```bash
-$ pulsar-admin sink localrun options
+$ pulsar-admin sinks localrun options
```
Options
@@ -1719,16 +1719,16 @@ Get the list of Pulsar IO connector sinks supported by Pulsar cluster
Usage
```bash
-$ pulsar-admin sink available-sinks
+$ pulsar-admin sinks available-sinks
```
-## `source`
+## `sources`
An interface for managing Pulsar IO sources (ingress data into Pulsar)
Usage
```bash
-$ pulsar-admin source subcommand
+$ pulsar-admin sources subcommand
```
Subcommands
@@ -1750,7 +1750,7 @@ Submit a Pulsar IO source connector to run in a Pulsar cluster
Usage
```bash
-$ pulsar-admin source create options
+$ pulsar-admin sources create options
```
Options
@@ -1779,7 +1779,7 @@ Update a already submitted Pulsar IO source connector
Usage
```bash
-$ pulsar-admin source update options
+$ pulsar-admin sources update options
```
Options
@@ -1808,7 +1808,7 @@ Stops a Pulsar IO source connector
Usage
```bash
-$ pulsar-admin source delete options
+$ pulsar-admin sources delete options
```
Options
@@ -1824,7 +1824,7 @@ Gets the information about a Pulsar IO source connector
Usage
```bash
-$ pulsar-admin source get options
+$ pulsar-admin sources get options
```
Options
@@ -1840,7 +1840,7 @@ Check the current status of a Pulsar Source
Usage
```bash
-$ pulsar-admin source status options
+$ pulsar-admin sources status options
```
Options
@@ -1857,7 +1857,7 @@ List all running Pulsar IO source connectors
Usage
```bash
-$ pulsar-admin source list options
+$ pulsar-admin sources list options
```
Options
@@ -1872,7 +1872,7 @@ Stop source instance
Usage
```bash
-$ pulsar-admin source stop options
+$ pulsar-admin sources stop options
```
Options
@@ -1889,7 +1889,7 @@ Start source instance
Usage
```bash
-$ pulsar-admin source start options
+$ pulsar-admin sources start options
```
Options
@@ -1906,7 +1906,7 @@ Restart source instance
Usage
```bash
-$ pulsar-admin source restart options
+$ pulsar-admin sources restart options
```
Options
@@ -1923,7 +1923,7 @@ Run a Pulsar IO source connector locally (rather than deploying it to the Pulsar
Usage
```bash
-$ pulsar-admin source localrun options
+$ pulsar-admin sources localrun options
```
Options
@@ -1959,7 +1959,7 @@ Get the list of Pulsar IO connector sources supported by Pulsar cluster
Usage
```bash
-$ pulsar-admin source available-sources
+$ pulsar-admin sources available-sources
```
diff --git a/site2/docs/sql-getting-started.md b/site2/docs/sql-getting-started.md
index dd7dff3..05ae7a4 100644
--- a/site2/docs/sql-getting-started.md
+++ b/site2/docs/sql-getting-started.md
@@ -72,7 +72,7 @@ Splits: 19 total, 19 done (100.00%)
Currently, there is no data in Pulsar that we can query. Lets start the built-in connector _DataGeneratorSource_ to ingest some mock data for us to query:
```bash
-./bin/pulsar-admin source create --name generator --destinationTopicName generator_test --source-type data-generator
+./bin/pulsar-admin sources create --name generator --destinationTopicName generator_test --source-type data-generator
```
Afterwards, the will be a topic with can query in the namespace "public/default":