You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by zh...@apache.org on 2022/03/02 03:20:49 UTC
[rocketmq-connect] 41/43: Update README.md (#553)
This is an automated email from the ASF dual-hosted git repository.
zhoubo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-connect.git
commit 5f40a886bb415d1bd84b056c41027de9666d6c65
Author: Han.G <gr...@gmail.com>
AuthorDate: Mon Feb 22 12:23:04 2021 +0100
Update README.md (#553)
The config infomation has benn verified. This update could be a draft version config template in README.md and some parts of it will probobaly be changed in the close future.
---
README.md | 83 ++++++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 53 insertions(+), 30 deletions(-)
diff --git a/README.md b/README.md
index 747e718..54c04ea 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
## rocketmq-connect-jdbc 打包
```
-mvn clean install -DskipTest -U
+mvn clean install -Dmaven.test.skip=true
```
## rocketmq-connect-jdbc 启动
@@ -11,16 +11,33 @@ mvn clean install -DskipTest -U
```
http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-source-connector-name}
-?config={"connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSourceConnector",“dbUrl”:"${source-db-ip}",dbPort”:"${source-db-port}",dbUsername”:"${source-db-username}",dbPassword”:"${source-db-password}","rocketmqTopic":"jdbcTopic","mode":"bulk","whiteDataBase":{"${source-db-name}":{"${source-table-name}":{"${source-column-name}":"${source-column-value}"}}},"source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter"}
+?config={"source-rocketmq":"${runtime-ip}:${runtime-port}","source-cluster":"${broker-cluster}","connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSourceConnector",“dbUrl”:"${source-db-ip}",dbPort”:"${source-db-port}",dbUsername”:"${source-db-username}",dbPassword”:"${source-db-password}","rocketmqTopic":"${source-table-name}","mode":"bulk","whiteDataBase":{"${source-db-name}":{"${source-table-name}":{"${source-column-name}":"${source-column-value}"}}},"source-record-conve [...]
+```
+
+例子
+
+```
+http://localhost:8081/connectors/jdbcConnectorSource?config={"source-rocketmq":"localhost:9876","source-cluster":"DefaultCluster",
+"connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSourceConnector","dbUrl":"192.168.1.3","dbPort":"3306","dbUsername":"root","dbPassword":"mysqldb123456",
+"rocketmqTopic":"test_table","mode":"bulk","whiteDataBase":{"test_database":{"test_table":{"test_column":"8"}}},
+"source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter"}
```
* **jdbc-sink-connector** 启动
```
http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-sink-connector-name}
-?config={"connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSinkConnector",“dbUrl”:"${sink-db-ip}",dbPort”:"${sink-db-port}",dbUsername”:"${sink-db-username}",dbPassword”:"${sink-db-password}","rocketmqTopic":"jdbcTopic","mode":"bulk","topicNames":"${sink-topic-name}","source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter"}
+?config={"source-rocketmq":"${runtime-ip}:${runtime-port}","source-cluster":"${broker-cluster}","connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSinkConnector",“dbUrl”:"${sink-db-ip}",dbPort”:"${sink-db-port}",dbUsername”:"${sink-db-username}",dbPassword”:"${sink-db-password}","mode":"bulk","topicNames":"${source-table-name}","source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter"}
```
->**注：** `rocketmq-jdbc-connect` 的启动依赖于`rocketmq-connect-runtime`项目的启动，需将打好的`jar`包放置到`runtime`项目中`pluginPaths`配置的路径后再执行上面的启动请求,该值配置在`runtime`项目下的`connect.conf`文件中
+
+例子
+```
+http://localhost:8081/connectors/jdbcConnectorSink?config={"source-rocketmq":"localhost:9876","source-cluster":"DefaultCluster",
+"connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSinkConnector","dbUrl":"192.168.1.2","dbPort":"3306","dbUsername":"root",
+"dbPassword":"mysqldb123456","topicNames":"test_table","mode":"bulk","source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter"}
+```
+
+>**注：** `rocketmq-jdbc-connect` 的启动依赖于`rocketmq-connect-runtime`项目的启动，需将打好的所有`jar`包放置到`runtime`项目中`pluginPaths`配置的路径后再执行上面的启动请求,该值配置在`runtime`项目下的`connect.conf`文件中
## rocketmq-connect-jdbc 停止
@@ -31,32 +48,38 @@ http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-connector-name}/
## rocketmq-connect-jdbc 参数说明
* **jdbc-source-connector 参数说明**
-参数 | 类型 | 是否必须 | 描述 | 样例
-|---|---|---|---|---|
-|dbUrl | String | 是 | source端 DB ip | 192.168.1.2|
-|dbPort | String | 是 | source端 DB port | 3306 |
-|dbUsername | String | 是 | source端 DB 用户名 | root |
-|dbPassword | String | 是 | source端 DB 密码 | 123456 |
-|whiteDataBase | String | 是 | source端同步数据白名单，嵌套配置，为{DB名：{表名：{字段名：字段值}}}，若无指定字段数据同步，字段名可设为NO-FILTER，值为任意 | {"DATABASE_TEST":{"TEST_DATA":{"name":"test"}}} |
-|mode | String | 是 | source-connector 模式，目前仅支持bulk | bulk |
-|~~rocketmqTopic~~ | String | 是 | 待废弃 | jdbcTopic |
-|task-divide-strategy | Integer | 否 | task 分配策略, 默认值为 0，表示按照topic分配任务，每一个table便是一个topic | 0 |
-|task-parallelism | Integer | 否 | task parallelism，默认值为 1，表示将topic拆分为多少个任务进行执行 | 2 |
-|source-record-converter | String | 是 | source data 解析 | org.apache.rocketmq.connect.runtime.converter.JsonConverter |
+| KEY | TYPE | Must be filled | Description| Example
+|------------------------|---------|----------------|------------|---|
+|dbUrl | String | YES | source端 DB ip | 192.168.1.3|
+|dbPort | String | YES | source端 DB port | 3306 |
+|dbUsername | String | YES | source端 DB 用户名 | root |
+|dbPassword | String | YES | source端 DB 密码 | 123456 |
+|whiteDataBase | String | YES | source端同步数据白名单，嵌套配置，为{DB名：{表名：{字段名：字段值}}}，若无指定字段数据同步，字段名可设为NO-FILTER，值为任意 | {"DATABASE_TEST":{"TEST_DATA":{"name":"test"}}} |
+|mode | String | YES | source-connector 模式，目前仅支持bulk | bulk |
+|rocketmqTopic | String | NO | source端同步数据的topic名字，必须和要同步的数据库表名一样 | TEST_DATA |
+|task-divide-strategy | Integer | NO | task 分配策略, 默认值为 0，表示按照topic分配任务，每一个table便是一个topic | 0 |
+|task-parallelism | Integer | NO | task parallelism，默认值为 1，表示将topic拆分为多少个任务进行执行 | 2 |
+|source-rocketmq | String | YES | source 端获取路由信息连接到的RocketMQ nameserver 地址 | 192.168.1.3:9876 |
+|source-cluster | String | YES | source 端获取路由信息连接到的RocketMQ broker cluster | DefaultCluster |
+|source-record-converter | String | YES | source data 解析 | org.apache.rocketmq.connect.runtime.converter.JsonConverter |
+```
+注：1. source/sink配置文件说明是以rocketmq-connect-jdcb为demo，不同source/sink connector配置有差异，请以具体sourc/sink connector为准
+ 2. rocketmqTopic 在jdbc-source-connector中没有被用到，暂时保留的原因是为了配置显示一致性
+```
* **jdbc-sink-connector 参数说明**
-参数 | 类型 | 是否必须 | 描述 | 样例
-|---|---|---|---|---|
-|dbUrl | String | 是 | sink端 DB ip | 192.168.1.2|
-|dbPort | String | 是 | sink端 DB port | 3306 |
-|dbUsername | String | 是 | sink端 DB 用户名 | root |
-|dbPassword | String | 是 | sink端 DB 密码 | 123456 |
-|topicNames | String | 是 | sink端同步数据的topic名字 | topic-1,topic-2 |
-|mode | String | 是 | source-connector 模式，目前仅支持bulk | bulk |
-|~~rocketmqTopic~~ | String | 是 | 待废弃 | jdbcTopic |
-|task-divide-strategy | Integer | 否 | task 分配策略, 默认值为 0，表示按照topic分配任务，每一个table便是一个topic | 0 |
-|task-parallelism | Integer | 否 | task parallelism，默认值为 1，表示将topic拆分为多少个任务进行执行 | 2 |
-|source-rocketmq | String | 是 | sink 端获取路由信息连接到的RocketMQ nameserver 地址 | TODO |
-|source-rocketmq | String | 是 | sink 端获取路由信息连接到的RocketMQ broker cluster 地址 | TODO |
-|source-record-converter | String | 是 | source data 解析 | org.apache.rocketmq.connect.runtime.converter.JsonConverter |
+| KEY | TYPE | Must be filled | Description| Example
+|------------------------|---------|----------------|------------|---|
+|dbUrl | String | YES | sink端 DB ip | 192.168.1.2|
+|dbPort | String | YES | sink端 DB port | 3306 |
+|dbUsername | String | YES | sink端 DB 用户名 | root |
+|dbPassword | String | YES | sink端 DB 密码 | 123456 |
+|mode | String | YES | source-connector 模式，目前仅支持bulk | bulk |
+|topicNames | String | YES | sink端同步数据的topic名字，必须和要同步的数据库表名一样 | TEST_DATA |
+|task-divide-strategy | Integer | NO | sink端 分配策略, 默认值为 0，表示按照topic分配任务，每一个table便是一个topic | 0 |
+|task-parallelism | Integer | NO | sink端 parallelism，默认值为 1，表示将topic拆分为多少个任务进行执行 | 2 |
+|source-rocketmq | String | YES | sink端 端获取路由信息连接到的RocketMQ nameserver 地址 | 192.168.1.3:9876 |
+|source-cluster | String | YES | sink端 端获取路由信息连接到的RocketMQ broker cluster | DefaultCluster |
+|source-record-converter | String | YES | sink端 data 解析 | org.apache.rocketmq.connect.runtime.converter.JsonConverter |
+