You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2023/03/14 02:14:14 UTC
[inlong-website] branch master updated: [INLONG-707][Doc] Add parameter description for enabling incremental snapshot for MongoDB CDC (#710)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 32b93997f6 [INLONG-707][Doc] Add parameter description for enabling incremental snapshot for MongoDB CDC (#710)
32b93997f6 is described below
commit 32b93997f6f112b4b999d561491cbc11640c1140
Author: emhui <11...@users.noreply.github.com>
AuthorDate: Tue Mar 14 10:14:08 2023 +0800
[INLONG-707][Doc] Add parameter description for enabling incremental snapshot for MongoDB CDC (#710)
---
docs/data_node/extract_node/mongodb-cdc.md | 4 ++++
.../current/data_node/extract_node/mongodb-cdc.md | 4 ++++
2 files changed, 8 insertions(+)
diff --git a/docs/data_node/extract_node/mongodb-cdc.md b/docs/data_node/extract_node/mongodb-cdc.md
index 4eca173ad3..dd7b1ff7c0 100644
--- a/docs/data_node/extract_node/mongodb-cdc.md
+++ b/docs/data_node/extract_node/mongodb-cdc.md
@@ -127,10 +127,14 @@ TODO: It will be supported in the future.
| connection.options | optional | (none) | String | The ampersand-separated [connection options](https://docs.mongodb.com/manual/reference/connection-string/#std-label-connections-connection-options) of MongoDB. eg. `replicaSet=test&connectTimeoutMS=300000` |
| copy.existing | optional | true | Boolean | Whether copy existing data from source collections. |
| copy.existing.queue.size | optional | 10240 | Integer | The max size of the queue to use when copying data. |
+| batch.size | optional | 1024 | Integer | The cursor batch size. |
| poll.max.batch.size | optional | 1024 | Integer | Maximum number of change stream documents to include in a single batch when polling for new data. |
| poll.await.time.ms | optional | 1000 | Integer | The amount of time to wait before checking for new results on the change stream. |
| heartbeat.interval.ms | optional | 0 | Integer | The length of time in milliseconds between sending heartbeat messages. Use 0 to disa |
| inlong.metric.labels | optional | (none) | String | Inlong metric label, format of value is groupId=`{groupId}`&streamId=`{streamId}`&nodeId=`{nodeId}`. |
+| scan.incremental.snapshot.enabled | optional | false | Boolean | Whether enable incremental snapshot. The incremental snapshot feature only supports after MongoDB 4.0. |
+| scan.incremental.snapshot.chunk.size.mb | optional | 64 | Integer | The chunk size mb of incremental snapshot. |
+| chunk-meta.group.size | optional | 1000 | Integer | The group size of chunk meta, if the meta size exceeds the group size, the meta will be divided into multiple groups. |
## Available Metadata
The following format metadata can be exposed as read-only (VIRTUAL) columns in a table definition.
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md
index c3b94c189b..b6ad2b5794 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md
@@ -127,10 +127,14 @@ TODO: 未来会支持
| connection.options | 可选 | (none) | String | MongoDB的 & 分隔[连接选项](https://docs.mongodb.com/manual/reference/connection-string/#std-label-connections-connection-options)。例如。`replicaSet=test&connectTimeoutMS=300000` |
| copy.existing | 可选 | true | Boolean | 是否从源集合中复制现有数据。 |
| copy.existing.queue.size | 可选 | 10240 | Integer | 执行数据复制时使用的线程数。 |
+| batch.size | 可选 | 1024 | Integer | Cursor 批次大小。 |
| poll.max.batch.size | 可选 | 1024 | Integer | 轮询新数据时，单个批次中包含的最大更改流文档数。 |
| poll.await.time.ms | 可选 | 1000 | Integer | 在更改流上检查新结果之前等待的时间量。 |
| heartbeat.interval.ms | 可选 | 0 | Integer | 发送心跳消息之间的时间长度（以毫秒为单位）。使用 0 禁用。 |
| inlong.metric.labels | 可选 | (none) | String | inlong metric 的标签值，该值的构成为groupId=`{groupId}`&streamId=`{streamId}`&nodeId=`{nodeId}`。|
+| scan.incremental.snapshot.enabled | 可选 | false | Boolean | 是否启用增量快照。增量快照功能仅支持 MongoDB 4.0之后的版本。 |
+| scan.incremental.snapshot.chunk.size.mb | 可选 | 64 | Integer | 增量快照的块大小，单位： mb。 |
+| chunk-meta.group.size | 可选 | 1000 | Integer | chunk meta 的组大小，如果 meta 大小超过组大小，则 meta 将被分成多个组。 |
## 可用元数据