You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2022/06/16 04:50:36 UTC
[incubator-inlong-website] branch master updated: [INLONG-413][Sort] Update sort deployment doc (#414)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 7bc714f05 [INLONG-413][Sort] Update sort deployment doc (#414)
7bc714f05 is described below
commit 7bc714f0597715a1f66e7c63a6afe6a9b5dd4d56
Author: pacino <ge...@gmail.com>
AuthorDate: Thu Jun 16 12:50:30 2022 +0800
[INLONG-413][Sort] Update sort deployment doc (#414)
---
docs/modules/sort/quick_start.md | 77 +++++++++++++++-------
.../current/modules/sort/quick_start.md | 71 ++++++++++++++------
2 files changed, 103 insertions(+), 45 deletions(-)
diff --git a/docs/modules/sort/quick_start.md b/docs/modules/sort/quick_start.md
index e898ceeaa..d1a503bdb 100644
--- a/docs/modules/sort/quick_start.md
+++ b/docs/modules/sort/quick_start.md
@@ -4,42 +4,71 @@ sidebar_position: 2
---
## Set up Flink Environment
-Currently InLong-Sort is based on Flink, before you run an InLong-Sort Application,
+Currently, InLong-Sort is based on Flink, before you run an InLong-Sort Application,
you need to set up [Flink Environment](https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/overview/).
-Currently, InLong-Sort relys on Flink-1.13.5. Chose `flink-1.13.5-bin-scala_2.11.tgz` when downloading package.
+Currently, InLong-Sort relies on Flink-1.13.5. Chose `flink-1.13.5-bin-scala_2.11.tgz` when downloading package.
Once your Flink Environment is set up, you can visit Web UI of Flink, whose address is stored in `/${your_flink_path}/conf/masters`.
## Prepare installation files
-All installation files at `inlong-sort` directory.
+We need `sort-dist-[version].jar` and `sort-connector-[database]-[version].jar`.
-## Starting an inlong-sort application
-Now you can submit job to Flink with the jar compiled, refer to [How to submit job to Flink](https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/cli/#submitting-a-job).
+`sort-dist-[version].jar` include main class `org.apache.inlong.sort.Entrance`.
-Example：
-```
-./bin/flink run -c org.apache.inlong.sort.Entrance inlong-sort/sort-[version].jar \
---group.info.file /YOUR_DATASTREAM_DIR/mysql-to-kafka.json
-```
+`sort-connector-[database]-[version].jar` are connector jar.
-Notice：
+Please choose required connector jar by your data integration requirement.
-- `-c org.apache.inlong.sort.Entrance` is the main class name
+[Download](https://inlong.apache.org/download/main) `sort-dist-[version].jar` from `inlong-sort` of `apache-inlong-[version]-bin.tar.gz`.
-- `inlong-sort/sort-[version].jar` is the compiled jar
+[Download](https://inlong.apache.org/download/main) `sort-connector-[database]-[version].jar` from `apache-inlong-[version]-sort-connectors.tar.gz`.
-## Necessary configurations
-- `--group.info.file` data stream configuration file path
+Please put required jars into under `FLINK_HOME/lib/` after download.
-**Example**
-```
---group.info.file /YOUR_DATASTREAM_INFO_DIR/mysql-to-kafka.json
+## Starting an inlong-sort application
+Now you can submit job to Flink with the jar compiled, refer to [How to submit job to Flink](https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/cli/#submitting-a-job).
+
+Example：
+```shell
+./bin/flink run -c org.apache.inlong.sort.Entrance FLINK_HOME/lib/sort-dist-[version].jar \
+--sql.script.file /YOUR_SQL_SCRIPT_DIR/mysql-to-postgresql.sql
```
-## All configurations
-| name | necessary | default value | description |
-|--------------------------------------------|:---------:|:-------------:|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| checkpoint.interval | N | 600000 | checkpoint interval，unit: ms |
-| min.pause.between.checkpoints.ms | N | 500 | the minimal checkpoint interval, unit：ms |
-| checkpoint.timeout.ms | N | 600000 | checkpoint timeout，unit: ms |
\ No newline at end of file
+## Configuration
+`/YOUR_SQL_SCRIPT_DIR/mysql-to-postgresql.sql` is a sql script file includes multi Flink SQL statements that can be separated by semicolon.
+Statement can support `CREATE TABLE`, `CRETAE VIEW`, `INSERT INTO`. We can write sql to do data integration.
+
+We can write following SQL script if we want to read data from MySQL and write into PostgreSQL.
+```sql
+ CREATE TABLE `table_1`(
+ `age` INT,
+ `name` STRING)
+ WITH (
+ 'connector' = 'mysql-cdc-inlong',
+ 'hostname' = 'localhost',
+ 'username' = 'root',
+ 'password' = 'inlong',
+ 'database-name' = 'test',
+ 'scan.incremental.snapshot.enabled' = 'false',
+ 'server-time-zone' = 'GMT+8',
+ 'table-name' = 'user'
+);
+CREATE TABLE `table_2`(
+ PRIMARY KEY (`name`) NOT ENFORCED,
+ `name` STRING,
+ `age` INT)
+ WITH (
+ 'connector' = 'jdbc',
+ 'url' = 'jdbc:postgresql://localhost:5432/postgres',
+ 'username' = 'postgres',
+ 'password' = 'inlong',
+ 'table-name' = 'public.user',
+ 'port' = '3306'
+);
+INSERT INTO `table_2`
+ SELECT
+ `name` AS `name`,
+ `age` AS `age`
+ FROM `table_1`;
+```
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/sort/quick_start.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/sort/quick_start.md
index a6c41196b..49c693691 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/sort/quick_start.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/sort/quick_start.md
@@ -11,34 +11,63 @@ sidebar_position: 2
Flink 环境配置完成后，可以通过浏览器访问 Flink 的 Web UI，对应的地址是`/{Flink 部署路径}/conf/masters`文件中的地址
## 准备安装文件
-安装文件在`inlong-sort`目录。
+我们需要`sort-dist-[version].jar`和`sort-connector-[database]-[version].jar`。
-## 启动 InLong-Sort 应用
-有了上述编译阶段产出的jar包后，就可以启动 InLong-Sort 的应用了。提交方式可以参考[如何提交 Flink 作业](https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/cli/#submitting-a-job)。
+`sort-dist-[version].jar` 包含主类 `org.apache.inlong.sort.Entrance`。
-示例：
-```
-./bin/flink run -c org.apache.inlong.sort.Entrance inlong-sort/sort-[version].jar \
---group.info.file /YOUR_DATASTREAM_INFO_DIR/mysql-to-kafka.json
-```
+`sort-connector-[database]-[version].jar` 是连接器 jar。
-注意：
+请根据你的数据集成要求选择所需的连接器 jar。
-- `-c org.apache.inlong.sort.Entrance` 表示 Main class name
+请[下载](https://inlong.apache.org/zh-CN/download/main/) `apache-inlong-[version]-bin.tar.gz` 解压后从 `inlong-sort` 目录拿到 `sort-dist-[version].jar` 。
-- `inlong-sort/sort-[version].jar` 为编译阶段产出的 Jar 包
+请[下载](https://inlong.apache.org/zh-CN/download/main/) `apache-inlong-{version}-sort-connectors.tar.gz` 解压后拿到 `sort-connector-[database]-[version].jar`。
-## 必要的配置
-- `--group.info.file` 流配置文件路径
+下载后可以将需要的jars放到`FLINK_HOME/lib/`下。
-**启动参数配置示例**
+## 启动 InLong-Sort 应用
+有了上述编译阶段产出的jar包后，就可以启动 InLong-Sort 的应用了。提交方式可以参考[如何提交 Flink 作业](https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/cli/#submitting-a-job)。
+
+示例：
```
---group.info.file /YOUR_DATASTREAM_INFO_DIR/mysql-to-kafka.json
+./bin/flink run -c org.apache.inlong.sort.Entrance FLINK_HOME/lib/sort-dist-[version].jar \
+--sql.script.file /YOUR_SQL_SCRIPT_DIR/mysql-to-postgresql.sql
```
-## 所有支持的配置
-| 配置名 | 是否必须 | 默认值 | 描述 |
-|--------------------------------------------|:------:|:-----------:|---------------------------------------------------|
-| checkpoint.interval | N | 600000 | checkpoint间隔，单位：毫秒 |
-| min.pause.between.checkpoints.ms | N | 500 | checkpoint之间的最小间隔，单位：毫秒 |
-| checkpoint.timeout.ms | N | 600000 | checkpoint超时时间，单位：毫秒 |
\ No newline at end of file
+## 必要的配置
+`/YOUR_SQL_SCRIPT_DIR/mysql-to-postgresql.sql` 是一个 sql 脚本文件，包含多个 Flink SQL 语句，可以用分号分隔。
+语句可以支持`CREATE TABLE`、`CRETAE VIEW`、`INSERT INTO`。 我们可以写sql来做数据集成。
+
+如果我们想从 MySQL 读取数据并写入 PostgreSQL，我们可以编写以下 SQL 脚本。
+```sql
+ CREATE TABLE `table_1`(
+ `age` INT,
+ `name` STRING)
+ WITH (
+ 'connector' = 'mysql-cdc-inlong',
+ 'hostname' = 'localhost',
+ 'username' = 'root',
+ 'password' = 'inlong',
+ 'database-name' = 'test',
+ 'scan.incremental.snapshot.enabled' = 'false',
+ 'server-time-zone' = 'GMT+8',
+ 'table-name' = 'user'
+);
+CREATE TABLE `table_2`(
+ PRIMARY KEY (`name`) NOT ENFORCED,
+ `name` STRING,
+ `age` INT)
+ WITH (
+ 'connector' = 'jdbc',
+ 'url' = 'jdbc:postgresql://localhost:5432/postgres',
+ 'username' = 'postgres',
+ 'password' = 'inlong',
+ 'table-name' = 'public.user',
+ 'port' = '3306'
+);
+INSERT INTO `table_2`
+ SELECT
+ `name` AS `name`,
+ `age` AS `age`
+ FROM `table_1`;
+```
\ No newline at end of file