[GitHub] [pulsar] murong00 opened a new pull request #3823: [Issue 3822] [pulsar-io] Add a Pulsar IO connector for Alluxio sink

[GitBox <gi...@apache.org>]

murong00 opened a new pull request #3823: [Issue 3822] [pulsar-io] Add a Pulsar IO connector for Alluxio sink
URL: https://github.com/apache/pulsar/pull/3823
 
 
   ### Motivation
   
   Fixes #3822
   
   Provides a builtin alluxio sink Connector, in order to enable a single common interface and namespace for data access in some application cases.
   
   ### Modifications
   
   Add a new sub-module in the `pulsar-io` module.
   
   ### Verifying this change
   
   This change can be verified as follows:
   
   * deploy the alluxio sink connector with configuration file containing the following fields:
   
   ```
   configs:
       alluxioMasterHost: "localhost"
       alluxioMasterPort: "19998"
       alluxioDir: "/pulsar"
       filePrefix: "pulsar"
       fileExtension: ".txt"
       lineSeparator: "\n"
       rotationRecords: "100"
       rotationInterval: "-1"
   ```
   * deploy an alluxio cluster and start the master and workers
   * send messages in the topic declared when deploying the connector
   * use the [`Alluxio CLI`](https://www.alluxio.org/docs/1.8/en/basic/Command-Line-Interface.html) to check if the messages have been stored in the specified alluxio directory.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services