You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ur...@apache.org on 2022/12/10 17:11:36 UTC
[pulsar-site] branch main updated: Docs sync done from apache/pulsar (#b0695fb)
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new a590c06d541 Docs sync done from apache/pulsar (#b0695fb)
a590c06d541 is described below
commit a590c06d5412279e78f57a8e9a512165aa859c99
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Sat Dec 10 17:11:30 2022 +0000
Docs sync done from apache/pulsar (#b0695fb)
---
site2/website-next/docs/admin-api-topics.md | 24 ++++-----
site2/website-next/docs/administration-stats.md | 70 +++++++++++++------------
site2/website-next/docs/io-alluxio.md | 62 ++++++++++++++++++++++
3 files changed, 110 insertions(+), 46 deletions(-)
diff --git a/site2/website-next/docs/admin-api-topics.md b/site2/website-next/docs/admin-api-topics.md
index a46d93be310..87c16345b32 100644
--- a/site2/website-next/docs/admin-api-topics.md
+++ b/site2/website-next/docs/admin-api-topics.md
@@ -12,14 +12,14 @@ import TabItem from '@theme/TabItem';
:::tip
- This page only shows **some frequently used operations**.
-
- - For the latest and complete information about `Pulsar admin`, including commands, flags, descriptions, and more, see [Pulsar admin doc](/tools/pulsar-admin/)
-
- - For the latest and complete information about `REST API`, including parameters, responses, samples, and more, see {@inject: rest:REST:/} API doc.
-
- - For the latest and complete information about `Java admin API`, including classes, methods, descriptions, and more, see [Java admin API doc](/api/admin/).
-
+This page only shows **some frequently used operations**.
+
+- For the latest and complete information about `Pulsar admin`, including commands, flags, descriptions, and more, see [Pulsar admin doc](/tools/pulsar-admin/)
+
+- For the latest and complete information about `REST API`, including parameters, responses, samples, and more, see {@inject: rest:REST:/} API doc.
+
+- For the latest and complete information about `Java admin API`, including classes, methods, descriptions, and more, see [Java admin API doc](/api/admin/).
+
:::
Pulsar has persistent and non-persistent topics. A persistent topic is a logical endpoint for publishing and consuming messages. The topic name structure for persistent topics is:
@@ -593,7 +593,7 @@ admin.topics().examineMessage(topic, "latest", 1);
</Tabs>
````
-### Get message ID
+### Get message ID
You can get message ID published at or just after the given datetime.
@@ -1641,7 +1641,7 @@ admin.topics().updatePartitionedTopic(topic, numPartitions);
````
### Delete
-You can delete partitioned topics with the [`delete-partitioned-topic`](/tools/pulsar-admin/) command, REST API and Java.
+You can delete partitioned topics with the [`delete-partitioned-topic`](/tools/pulsar-admin/) command, REST API and Java.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1951,9 +1951,9 @@ admin.topics().getSubscriptions(topic);
</Tabs>
````
-### Unsubscribe subscription
+### Unsubscribe subscription
-When a subscription does not process messages anymore, you can unsubscribe it using one of the following methods.
+When a subscription does not process messages anymore, you can unsubscribe it using one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
diff --git a/site2/website-next/docs/administration-stats.md b/site2/website-next/docs/administration-stats.md
index ec2d6dd1202..ff6a5588a0a 100644
--- a/site2/website-next/docs/administration-stats.md
+++ b/site2/website-next/docs/administration-stats.md
@@ -37,19 +37,19 @@ The following table outlines the stats of a topic. For more details about how to
### Producer stats
-|Stat|Description|
-|---|---|
-|producerId| The internal identifier for this producer on this topic.|
-|producerName| The internal identifier for this producer, generated by the client library.|
-|address| IP address and source port for the connection of this producer.|
-|connectedSince| The timestamp when this producer is created or reconnected last time.|
-|accessMode| The type of access to the topic that the producer requires.|
-|msgRateIn| The total rate of messages (message per second) published by this publisher.|
-|msgThroughputIn| The total throughput (byte per second) of the messages published by this publisher.|
-|averageMsgSize| The average message size in bytes from this publisher within the last interval.|
-|chunkedMessageRate| The total rate of chunked messages published by this publisher.|
-|clientVersion| The client library version of this producer.|
-|metadata| The metadata (key/value strings) associated with this publisher.|
+| Stat | Description |
+|--------------------|-------------------------------------------------------------------------------------|
+| producerId | The internal identifier for this producer on this topic. |
+| producerName | The internal identifier for this producer, generated by the client library. |
+| address | IP address and source port for the connection of this producer. |
+| connectedSince | The timestamp when this producer is created or reconnected last time. |
+| accessMode | The type of access to the topic that the producer requires. |
+| msgRateIn | The total rate of messages (message per second) published by this publisher. |
+| msgThroughputIn | The total throughput (byte per second) of the messages published by this publisher. |
+| averageMsgSize | The average message size in bytes from this publisher within the last interval. |
+| chunkedMessageRate | The total rate of chunked messages published by this publisher. |
+| clientVersion | The client library version of this producer. |
+| metadata | The metadata (key/value strings) associated with this publisher. |
### Subscription stats
@@ -93,27 +93,29 @@ The following table outlines the stats of a topic. For more details about how to
### Consumer stats
-| Stat | Description |
-|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| consumerName | Internal identifier for this consumer, generated by the client library. |
-| msgRateOut | The total rate of messages (message per second) delivered to the consumer. |
-| msgThroughputOut | The total throughput (byte per second) delivered to the consumer. |
-| availablePermits | The number of messages that the consumer has space for in the client library's listening queue. `0` means the client library's queue is full and `receive()` isn't called. A non-zero value means this consumer is ready for dispatched messages. |
-| unackedMessages | The number of unacknowledged messages for the consumer, where an unacknowledged message has been sent to the consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgment. |
-| blockedConsumerOnUnackedMsgs | The flag used to verify if the consumer is blocked due to reaching the threshold of the unacknowledged messages. |
-| lastConsumedTimestamp | The timestamp when the consumer reads a message the last time. |
-| lastAckedTimestamp | The timestamp when the consumer acknowledges a message the last time. |
-| address | The IP address and source port for the connection of this consumer. |
-| connectedSince | The timestamp when this consumer is created or reconnected last time. |
-| clientVersion | The client library version of this consumer. |
-| bytesOutCounter | The total bytes delivered to a consumer. |
-| msgOutCounter | The total messages delivered to a consumer. |
-| msgRateRedeliver | The total rate of messages redelivered by this consumer (message per second). |
-| chunkedMessageRate | The total rate of chunked messages delivered to this consumer. |
-| avgMessagesPerEntry | The number of average messages per entry for the consumer consumed. |
-| readPositionWhenJoining | The read position of the cursor when the consumer joins. |
-| keyHashRanges | The hash ranges assigned to this consumer if it uses Key_Shared sub mode. |
-| metadata | The metadata (key/value strings) associated with this consumer. |
+| Stat | Description |
+|------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| consumerName | Internal identifier for this consumer, generated by the client library. |
+| msgRateOut | The total rate of messages (message per second) delivered to the consumer. |
+| msgThroughputOut | The total throughput (byte per second) delivered to the consumer. |
+| availablePermits | The number of messages that the consumer has space for in the client library's listening queue. `0` means the client library's queue is full and `receive()` isn't called. A non-zero value means this consumer is ready for dispatched messages. |
+| unackedMessages | The number of unacknowledged messages for the consumer, where an unacknowledged message has been sent to the consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgment. |
+| blockedConsumerOnUnackedMsgs | The flag used to verify if the consumer is blocked due to reaching the threshold of the unacknowledged messages. |
+| lastConsumedTimestamp (Deprecated) | The timestamp when the consumer reads a message the last time. |
+| lastConsumedTime | The time when the consumer reads a message the last time. |
+| lastAckedTimestamp (Deprecated) | The timestamp when the consumer acknowledges a message the last time. |
+| lastAckedTime | The time when the consumer acknowledges a message the last time. |
+| address | The IP address and source port for the connection of this consumer. |
+| connectedSince | The timestamp when this consumer is created or reconnected last time. |
+| clientVersion | The client library version of this consumer. |
+| bytesOutCounter | The total bytes delivered to a consumer. |
+| msgOutCounter | The total messages delivered to a consumer. |
+| msgRateRedeliver | The total rate of messages redelivered by this consumer (message per second). |
+| chunkedMessageRate | The total rate of chunked messages delivered to this consumer. |
+| avgMessagesPerEntry | The number of average messages per entry for the consumer consumed. |
+| readPositionWhenJoining | The read position of the cursor when the consumer joins. |
+| keyHashRanges | The hash ranges assigned to this consumer if it uses Key_Shared sub mode. |
+| metadata | The metadata (key/value strings) associated with this consumer. |
### Replication stats
diff --git a/site2/website-next/docs/io-alluxio.md b/site2/website-next/docs/io-alluxio.md
new file mode 100644
index 00000000000..59c3247f9e6
--- /dev/null
+++ b/site2/website-next/docs/io-alluxio.md
@@ -0,0 +1,62 @@
+---
+id: io-alluxio
+title: Alluxio sink connector
+sidebar_label: Alluxio sink connector
+---
+
+## Sink
+
+The Alluxio sink connector pulls messages from Pulsar topics and persists the messages to an Alluxio directory.
+
+## Configuration
+
+The configuration of the Alluxio sink connector has the following properties.
+
+### Property
+
+| Name | Type|Required | Default | Description
+|------|----------|----------|---------|-------------|
+| `alluxioMasterHost` | String | true | "" (empty string) | The hostname of Alluxio master. |
+| `alluxioMasterPort` | int | true | 19998 | The port that Alluxio master node runs on. |
+| `alluxioDir` | String | true | "" (empty string) | The Alluxio directory from which files should be read from or written to. |
+| `securityLoginUser` | String | false | "" (empty string) | When `alluxio.security.authentication.type` is set to `SIMPLE` or `CUSTOM`, user application uses this property to indicate the user requesting Alluxio service. If it is not set explicitly, the OS login user is used. |
+| `filePrefix` | String | false | "" (empty string) | The prefix of the files to create in the Alluxio directory (e.g. a value of 'TopicA' results in files named topicA-, topicA-, etc being produced). |
+| `fileExtension` | String | false | "" (empty string) | The extension to add to the files written to Alluxio (e.g. '.txt'). |
+| `lineSeparator` | String | false | "" (empty string) | The character used to separate records in a text file. If no value is provided, then the content from all of the records is concatenated together in one continuous byte array. |
+| `rotationRecords` | long | false | 10000 | The number records of Alluxio file rotation. |
+| `rotationInterval` | long | false | -1 | The interval to rotate a Alluxio file (in milliseconds). |
+| `schemaEnable` | boolean | false | false | Sets whether the Sink has to take into account the Schema or if it should simply copy the raw message to Alluxio. |
+| `writeType` | String | false | `MUST_CACHE` | Default write type when creating Alluxio files. Valid options are `MUST_CACHE` (write only goes to Alluxio and must be stored in Alluxio), `CACHE_THROUGH` (try to cache, write to UnderFS synchronously), `THROUGH` (no cache, write to UnderFS synchronously). |
+
+### Example
+
+Before using the Alluxio sink connector, you need to create a configuration file in the path you will start Pulsar service (i.e. `PULSAR_HOME`) through one of the following methods.
+
+* JSON
+
+ ```json
+ {
+ "alluxioMasterHost": "localhost",
+ "alluxioMasterPort": "19998",
+ "alluxioDir": "pulsar",
+ "filePrefix": "TopicA",
+ "fileExtension": ".txt",
+ "lineSeparator": "\n",
+ "rotationRecords": "100",
+ "rotationInterval": "-1"
+ }
+ ```
+
+* YAML
+
+ ```yaml
+ configs:
+ alluxioMasterHost: "localhost"
+ alluxioMasterPort: "19998"
+ alluxioDir: "pulsar"
+ filePrefix: "TopicA"
+ fileExtension: ".txt"
+ lineSeparator: "\n"
+ rotationRecords: 100
+ rotationInterval: "-1"
+ ```
\ No newline at end of file