You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/19 01:11:09 UTC
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #3823: [Issue 3822] [pulsar-io] Add a Pulsar IO connector for Alluxio sink
Anonymitaet commented on a change in pull request #3823:
URL: https://github.com/apache/pulsar/pull/3823#discussion_r691698673
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
Review comment:
```suggestion
title: Alluxio sink connector
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
+
+The alluxio Sink Connector is used to pull messages from Pulsar topics and persist the messages
+to an alluxio directory.
Review comment:
```suggestion
to an Alluxio directory.
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
Review comment:
```suggestion
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
Review comment:
```suggestion
sidebar_label: Alluxio sink connector
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
+
+The alluxio Sink Connector is used to pull messages from Pulsar topics and persist the messages
+to an alluxio directory.
+
+## Sink Configuration Options
Review comment:
```suggestion
## Configuration
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
+
+The alluxio Sink Connector is used to pull messages from Pulsar topics and persist the messages
Review comment:
```suggestion
The Alluxio sink connector pulls messages from Pulsar topics and persists the messages
```
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
Review comment:
keep consistent with other connector docs
##########
File path: site2/docs/io-alluxio.md
##########
@@ -0,0 +1,25 @@
+---
+id: io-alluxio
+title: alluxio Connector
+sidebar_label: alluxio Connector
+---
+
+## Sink
+
+The alluxio Sink Connector is used to pull messages from Pulsar topics and persist the messages
+to an alluxio directory.
+
+## Sink Configuration Options
+
+| Name | Default | Required | Description |
+|------|---------|----------|-------------|
+| `alluxioMasterHost` | `null` | `true` | The hostname of Alluxio master. |
+| `alluxioMasterPort` | `19998` | `true` | The port that Alluxio master node runs on. |
+| `alluxioDir` | `null` | `true` | The alluxio directory from which files should be read from or written to. |
+| `securityLoginUser` | `null` | `false` | When alluxio.security.authentication.type is set to SIMPLE or CUSTOM, user application uses this property to indicate the user requesting Alluxio service. If it is not set explicitly, the OS login user will be used. |
+| `filePrefix` | `null` | `false` | The prefix of the files to create in the Alluxio directory (e.g. a value of 'TopicA' will result in files named topicA-, topicA-, etc being produced). |
+| `fileExtension` | `null` | `false` | The extension to add to the files written to Alluxio (e.g. '.txt'). |
+| `lineSeparator` | `null` | `false` | The character used to separate records in a text file. If no value is provided then the content from all of the records will be concatenated together in one continuous byte array. |
+| `rotationRecords` | `10000` | `false` | The number records of alluxio file rotation. |
+| `rotationInterval` | `-1` | `false` | The interval (in milliseconds) to rotate a alluxio file. |
+| `writeType` | `MUST_CACHE` | `false` | Default write type when creating Alluxio files. Valid options are `MUST_CACHE` (write will only go to Alluxio and must be stored in Alluxio), `CACHE_THROUGH` (try to cache, write to UnderFS synchronously), `THROUGH` (no cache, write to UnderFS synchronously). |
Review comment:
```suggestion
| `alluxioMasterHost` | `null` | `true` | The hostname of Alluxio master. |
| `alluxioMasterPort` | `19998` | `true` | The port that Alluxio master node runs on. |
| `alluxioDir` | `null` | `true` | The Alluxio directory from which files should be read from or written to. |
| `securityLoginUser` | `null` | `false` | When `alluxio.security.authentication.type` is set to SIMPLE or CUSTOM, user application uses this property to indicate the user requesting Alluxio service. If it is not set explicitly, the OS login user is used. |
| `filePrefix` | `null` | `false` | The prefix of the files to create in the Alluxio directory (e.g. a value of 'TopicA' results in files named topicA-, topicA-, etc being produced). |
| `fileExtension` | `null` | `false` | The extension to add to the files written to Alluxio (e.g. '.txt'). |
| `lineSeparator` | `null` | `false` | The character used to separate records in a text file. If no value is provided, then the content from all of the records is concatenated together in one continuous byte array. |
| `rotationRecords` | `10000` | `false` | The number records of Alluxio file rotation. |
| `rotationInterval` | `-1` | `false` | The interval to rotate Alluxio file (in milliseconds). |
| `writeType` | `MUST_CACHE` | `false` | Default write type when creating Alluxio files. Valid options are `MUST_CACHE` (write only goes to Alluxio and must be stored in Alluxio), `CACHE_THROUGH` (try to cache, write to UnderFS synchronously), `THROUGH` (no cache, write to UnderFS synchronously). |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org