You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2021/08/27 13:00:07 UTC

[incubator-inlong-website] branch master updated: [INLONG-1485][sort] add the guide documents for using Pulsar (#132)

This is an automated email from the ASF dual-hosted git repository.

dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git


The following commit(s) were added to refs/heads/master by this push:
     new aded00a  [INLONG-1485][sort] add the guide documents for using Pulsar (#132)
aded00a is described below

commit aded00a5ed0747ba3306d05b3cb54ac5d641235a
Author: Kevin Wen <89...@users.noreply.github.com>
AuthorDate: Fri Aug 27 20:59:59 2021 +0800

    [INLONG-1485][sort] add the guide documents for using Pulsar (#132)
---
 docs/en-us/modules/sort/introduction.md |  7 ++++---
 docs/en-us/modules/sort/quick_start.md  | 16 ++++++++--------
 docs/zh-cn/modules/sort/introduction.md |  3 ++-
 docs/zh-cn/modules/sort/quick_start.md  |  8 ++++----
 4 files changed, 18 insertions(+), 16 deletions(-)

diff --git a/docs/en-us/modules/sort/introduction.md b/docs/en-us/modules/sort/introduction.md
index a59e24f..948af31 100644
--- a/docs/en-us/modules/sort/introduction.md
+++ b/docs/en-us/modules/sort/introduction.md
@@ -6,7 +6,7 @@ Introduction - Apache InLong-sort
 Inlong-sort is used to extract data from different source systems, then transforms the data and finally loads the data into diffrent storage systems.
 Inlong-sort is simply an Flink application, and relys on Inlong-manager to manage meta data(such as the source informations and storage informations)
 
-#features
+# features
 ## multi-tenancy
 Inlong-sort is an multi-tenancy system, which means you can extract data from different sources(these sources must be of the same source type) and load data into different sinks(these sinks must be of the same storage type).
 e.g. you can extract data form different topics of inlong-tubemq and the load them to different hive clusters.
@@ -17,17 +17,18 @@ e.g if you want to change the schema of your data, just change the meta data on
 
 # supported sources
 - inlong-tubemq
+- pulsar
 
 # supported storages
 - clickhouse
 - hive (Currently we just support parquet file format)
 
-#limitations
+# limitations
 Currently, we just support extracting specified fields in the stage of **Transform**.
 
 # future plans
 ## More kinds of source systems
-pulsar, kafka and etc
+kafka and etc
 
 ## More kinds of storage systems
 Hbase, Elastic Search, and etc
diff --git a/docs/en-us/modules/sort/quick_start.md b/docs/en-us/modules/sort/quick_start.md
index 6205390..65add05 100644
--- a/docs/en-us/modules/sort/quick_start.md
+++ b/docs/en-us/modules/sort/quick_start.md
@@ -2,7 +2,7 @@
 Quick Start - Apache InLong-sort
 ---
 
-##  Set up flink environment
+## Set up flink environment
 Currently inlong-sort is based on flink, before you run an inlong-sort application,
 you need to set up flink environment.
 
@@ -12,10 +12,10 @@ Currently, inlong-sort relys on flink-1.9.3. Chose `flink-1.9.3-bin-scala_2.11.t
 
 Once your flink environment is set up, you can visit web ui of flink, whose address is stored in `/${your_flink_path}/conf/masters`.
 
-##  Prepare installation files
+## Prepare installation files
 All installation files at `inlong-sort` directory.
 
-##  Starting an inlong-sort application
+## Starting an inlong-sort application
 Now you can submit job to flink with the jar compiled.
 
 <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/deployment/yarn_setup.html#submit-job-to-flink" target="_blank">how to submit job to flink</a>
@@ -26,15 +26,15 @@ Example:
 
 Notice:
 
-- `-c org.apache.inlong.sort.flink.Entrance` 表示main class name
+- `-c org.apache.inlong.sort.flink.Entrance` is the main class name
 
-- `inlong-sort-core-1.0-SNAPSHOT.jar` 为编译阶段产出的jar包
+- `inlong-sort-core-1.0-SNAPSHOT.jar` is the compiled jar
 
-##  Necessary configurations
+## Necessary configurations
 - `--cluster-id ` which is used to represent a specified inlong-sort application
 - `--zookeeper.quorum` zk quorum
 - `--zookeeper.path.root` zk root path
-- `--source.type` source of the application, currently only "tubemq" is supported
+- `--source.type` source of the application, currently "tubemq" and "pulsar" are supported
 - `--sink.type` sink of the application, currently "clickhouse" and "hive" are supported
 
 Configurations above are necessary, you can see full configurations in
@@ -51,7 +51,7 @@ Configurations above are necessary, you can see full configurations in
 |cluster-id   |  Y | NA  |  used to represent a specified inlong-sort application |
 |zookeeper.quorum   | Y  | NA  | zk quorum  |
 |zookeeper.path.root   | Y  | "/inlong-sort"  |  zk root path  |
-|source.type   | Y | NA   | source of the application, currently only "tubemq" is supported  |
+|source.type   | Y | NA   | source of the application, currently "tubemq" and "pulsar" are supported  |
 |sink.type   | Y  | NA  | sink of the application, currently "clickhouse" and "hive" are supported  |
 |source.parallelism   | N  | 1  | parallelism of source  |
 |deserialization.parallelism   | N  |  1 | parallelism of deserialization  |
diff --git a/docs/zh-cn/modules/sort/introduction.md b/docs/zh-cn/modules/sort/introduction.md
index 3065e40..1fe29a3 100644
--- a/docs/zh-cn/modules/sort/introduction.md
+++ b/docs/zh-cn/modules/sort/introduction.md
@@ -20,6 +20,7 @@ inlong-sort支持热更新元数据,比如更新数据源的信息,数据sch
 
 # 支持的数据源
 - inlong-tubemq
+- pulsar
 
 # 支持的存储系统
 - hive(当前只支持parquet文件格式)
@@ -30,7 +31,7 @@ inlong-sort支持热更新元数据,比如更新数据源的信息,数据sch
 
 # 未来规划
 ## 支持更多种类的数据源
-pulsar,kafka等
+kafka等
 
 
 ## 支持更多种类的存储
diff --git a/docs/zh-cn/modules/sort/quick_start.md b/docs/zh-cn/modules/sort/quick_start.md
index bbf267a..7f830df 100644
--- a/docs/zh-cn/modules/sort/quick_start.md
+++ b/docs/zh-cn/modules/sort/quick_start.md
@@ -33,8 +33,8 @@ flink环境配置完成后,可以通过浏览器访问flink的web ui,对应
 - `--cluster-id ` 用来唯一标识一个inlong-sort作业
 - `--zookeeper.quorum` zk quorum
 - `--zookeeper.path.root` zk根目录
-- `--source.type` 数据源的种类, 当前仅支持"tubemq"
-- `--sink.type` 存储系统的种类,当前支持"clickhouse" 和 "hive"
+- `--source.type` 数据源的种类, 当前支持:"tubemq"、"pulsar"
+- `--sink.type` 存储系统的种类,当前支持:"clickhouse"、"hive"
 
 **配置示例**
 
@@ -46,8 +46,8 @@ flink环境配置完成后,可以通过浏览器访问flink的web ui,对应
 |cluster-id   | Y | NA  |  用来唯一标识一个inlong-sort作业 |
 |zookeeper.quorum   | Y  | NA  | zk quorum  |
 |zookeeper.path.root   | Y  | "/inlong-sort"  |  zk根目录  |
-|source.type   | Y | NA | 数据源的种类, 当前仅支持"tubemq"  |
-|sink.type   | Y  | NA  | 存储系统的种类,当前支持"clickhouse", "iceberg" 和 "hive" |
+|source.type   | Y | NA | 数据源的种类, 当前支持"tubemq"和"pulsar"  |
+|sink.type   | Y  | NA  | 存储系统的种类,当前支持"clickhouse" 和 "hive" |
 |source.parallelism   | N  | 1  | source的并行度  |
 |deserialization.parallelism | N | 1 | deserialization的并行度  |
 |sink.parallelism   | N  | 1  | sink的并行度 |