You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2023/03/14 02:08:46 UTC
[inlong-website] branch master updated: [INLONG-708][Doc] Describe Oracle CDC using Markdown (#711)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 4950c242ac [INLONG-708][Doc] Describe Oracle CDC using Markdown (#711)
4950c242ac is described below
commit 4950c242ac7de8b5de66c74b6894bd09127c2f32
Author: emhui <11...@users.noreply.github.com>
AuthorDate: Tue Mar 14 10:08:40 2023 +0800
[INLONG-708][Doc] Describe Oracle CDC using Markdown (#711)
---
docs/data_node/extract_node/oracle-cdc.md | 418 +++-----------------
.../current/data_node/extract_node/oracle-cdc.md | 420 +++------------------
2 files changed, 119 insertions(+), 719 deletions(-)
diff --git a/docs/data_node/extract_node/oracle-cdc.md b/docs/data_node/extract_node/oracle-cdc.md
index 4d868c10d7..80609954aa 100644
--- a/docs/data_node/extract_node/oracle-cdc.md
+++ b/docs/data_node/extract_node/oracle-cdc.md
@@ -236,172 +236,29 @@ TODO: It will be supported in the future.
## Oracle Extact Node Options
-<div class="highlight">
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left" style={{width: '25%'}}>Option</th>
- <th class="text-left" style={{width: '8%'}}>Required</th>
- <th class="text-left" style={{width: '7%'}}>Default</th>
- <th class="text-left" style={{width: '10%'}}>Type</th>
- <th class="text-left" style={{width: '50%'}}>Description</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>connector</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Specify what connector to use, here should be <code>'oracle-cdc-inlong'</code>.</td>
- </tr>
- <tr>
- <td>hostname</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>IP address or hostname of the Oracle database server.</td>
- </tr>
- <tr>
- <td>username</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Name of the Oracle database to use when connecting to the Oracle database server.</td>
- </tr>
- <tr>
- <td>password</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Password to use when connecting to the Oracle database server.</td>
- </tr>
- <tr>
- <td>database-name</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Database name of the Oracle server to monitor.</td>
- </tr>
- <tr>
- <td>schema-name</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Schema name of the Oracle database to monitor.</td>
- </tr>
- <tr>
- <td>table-name</td>
- <td>required</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Table name of the Oracle database to monitor. The value is of the form <i><schema_name>.<table_name></i></td>
- </tr>
- <tr>
- <td>port</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>1521</td>
- <td>Integer</td>
- <td>Integer port number of the Oracle database server.</td>
- </tr>
- <tr>
- <td>scan.startup.mode</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>initial</td>
- <td>String</td>
- <td>Optional startup mode for Oracle CDC consumer, valid enumerations are "initial"
- and "latest-offset".
- Please see <a href="#startup-reading-position">Startup Reading Position</a>section for more detailed information.</td>
- </tr>
- <tr>
- <td>debezium.*</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Pass-through Debezium's properties to Debezium Embedded Engine which is used to capture data changes from Oracle server.
- For example: <code>'debezium.snapshot.mode' = 'never'</code>.
- See more about the <a href="https://debezium.io/documentation/reference/1.5/connectors/oracle.html#oracle-connector-properties">Debezium's Oracle Connector properties</a></td>
- </tr>
- <tr>
- <td>inlong.metric.labels</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Inlong metric label, format of value is groupId=[groupId]&streamId=[streamId]&nodeId=[nodeId].</td>
- </tr>
- <tr>
- <td>source.multiple.enable</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>false</td>
- <td>Boolean</td>
- <td>Whether to enable multiple schema and table migration. If it is' true ', Oracle Extract Node will compress the physical field of the table into a special meta field 'data_canal' in the format of 'canal json'.</td>
- </tr>
- <tr>
- <td>scan.incremental.snapshot.enabled</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>true</td>
- <td>Boolean</td>
- <td>Incremental snapshot is a new mechanism to read snapshot of a table. Compared to the old snapshot mechanism, the incremental snapshot has many advantages, including: (1) source can be parallel during snapshot reading, (2) source can perform checkpoints in the chunk granularity during snapshot reading, (3) source doesn't need to acquire ROW SHARE MODE lock before snapshot reading.</td>
- </tr>
- <tr>
- <td>scan.incremental.snapshot.chunk.size</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>8096</td>
- <td>Integer</td>
- <td>The chunk size (number of rows) of table snapshot, captured tables are split into multiple chunks when read the snapshot of table.</td>
- </tr>
- <tr>
- <td>scan.snapshot.fetch.size</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>1024</td>
- <td>Integer</td>
- <td>The maximum fetch size for per poll when read table snapshot.</td>
- </tr>
- <tr>
- <td>connect.max-retries</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>3</td>
- <td>Integer</td>
- <td>The max retry times that the connector should retry to build Oracle database server connection.</td>
- </tr>
- <tr>
- <td>chunk-meta.group.size</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>1000</td>
- <td>Integer</td>
- <td>The group size of chunk meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.</td>
- </tr>
- <tr>
- <td>connect.timeout</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>30s</td>
- <td>Duration</td>
- <td>The maximum time that the connector should wait after trying to connect to the Oracle database server before timing out.</td>
- </tr>
- <tr>
- <td>chunk-key.even-distribution.factor.lower-bound</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>0.05d</td>
- <td>Double</td>
- <td>The lower bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount.</td>
- </tr>
- <tr>
- <td>chunk-key.even-distribution.factor.upper-bound</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>1000.0d</td>
- <td>Double</td>
- <td>The upper bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount.</td>
- </tr>
- <tr>
- <td>connection.pool.size</td>
- <td>optional</td>
- <td style={{wordWrap: 'break-word'}}>20</td>
- <td>Integer</td>
- <td>The connection pool size.</td>
- </tr>
- </tbody>
-</table>
-</div>
+| **Option** | **Required** | **Default** | **Type** | **Description** |
+| ------------------------- | ------------ | ---------- | -------- | ------------------------------------------------------------ |
+| connector| required| (none)| String| Specify what connector to use, here should be `oracle-cdc-inlong`.|
+| hostname| required| (none)| String| IP address or hostname of the Oracle database server.|
+| username| required| (none)| String| Name of the Oracle database to use when connecting to the Oracle database server.|
+| password| required| (none)| String| Password to use when connecting to the Oracle database server.|
+| database-name| required| (none)| String| Database name of the Oracle server to monitor.|
+| schema-name| required| (none)| String| Schema name of the Oracle database to monitor.|
+| table-name| required| (none)| String| Table name of the Oracle database to monitor. The value is of the form <i><schema_name>.<table_name></i>|
+| port| optional| 1521| Integer| Integer port number of the Oracle database server.|
+| scan.startup.mode| optional| initial| String| Optional startup mode for Oracle CDC consumer, valid enumerations are "initial" and "latest-offset". Please see <a href="#startup-reading-position">Startup Reading Position</a>section for more detailed information.|
+| debezium.*| optional| (none)| String| Pass-through Debezium's properties to Debezium Embedded Engine which is used to capture data changes from Oracle server. For example: <code>'debezium.snapshot.mode' = 'never'</code>. See more about the <a href="https://debezium.io/documentation/reference/1.5/connectors/oracle.html#oracle-connector-properties">Debezium's Oracle Connector properties</a>|
+| inlong.metric.labels| optional| (none)| String| Inlong metric label, format of value is groupId=[groupId]&streamId=[streamId]&nodeId=[nodeId].|
+| source.multiple.enable| optional| false| Boolean| Whether to enable multiple schema and table migration. If it is' true ', Oracle Extract Node will compress the physical field of the table into a special meta field 'data_canal' in the format of 'canal json'.|
+| scan.incremental.snapshot.enabled| optional| true| Boolean| Incremental snapshot is a new mechanism to read snapshot of a table. Compared to the old snapshot mechanism, the incremental snapshot has many advantages, including: (1) source can be parallel during snapshot reading, (2) source can perform checkpoints in the chunk granularity during snapshot reading, (3) source doesn't need to acquire ROW SHARE MODE lock before snapshot reading.|
+| scan.incremental.snapshot.chunk.size| optional| 8096| Integer| The chunk size (number of rows) of table snapshot, captured tables are split into multiple chunks when read the snapshot of table.|
+| scan.snapshot.fetch.size| optional| 1024| Integer| The maximum fetch size for per poll when read table snapshot.|
+| connect.max-retries| optional| 3| Integer| The max retry times that the connector should retry to build Oracle database server connection.|
+| chunk-meta.group.size| optional| 1000| Integer| The group size of chunk meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.|
+| connect.timeout| optional| 30s| Duration| The maximum time that the connector should wait after trying to connect to the Oracle database server before timing out.|
+| chunk-key.even-distribution.factor.lower-bound| optional| 0.05d| Double| The lower bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount.|
+| chunk-key.even-distribution.factor.upper-bound| optional| 1000.0d| Double| The upper bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount.|
+| connection.pool.size| optional| 20| Integer| The connection pool size.|
## Limitation
@@ -420,92 +277,23 @@ restart-strategy.fixed-delay.attempts: 2147483647
The following format metadata can be exposed as read-only (VIRTUAL) columns in a table definition.
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left" style={{width: '15%'}}>Key</th>
- <th class="text-left" style={{width: '30%'}}>DataType</th>
- <th class="text-left" style={{width: '55%'}}>Description</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>table_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the table that contain the row.</td>
- </tr>
- <tr>
- <td>schema_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the schema that contain the row.</td>
- </tr>
- <tr>
- <td>database_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the database that contain the row.</td>
- </tr>
- <tr>
- <td>op_ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>It indicates the time that the change was made in the database. <br/>If the record is read from snapshot of the table instead of the change stream, the value is always 0.</td>
- </tr>
- <tr>
- <td>meta.table_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the table that contain the row.</td>
- </tr>
- <tr>
- <td>meta.schema_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the schema that contain the row.</td>
- </tr>
- <tr>
- <td>meta.database_name</td>
- <td>STRING NOT NULL</td>
- <td>Name of the database that contain the row.</td>
- </tr>
- <tr>
- <td>meta.op_ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>It indicates the time that the change was made in the database. <br/>If the record is read from snapshot of the table instead of the change stream, the value is always 0.</td>
- </tr>
- <tr>
- <td>meta.op_type</td>
- <td>STRING</td>
- <td>Type of database operation, such as INSERT/DELETE, etc.</td>
- </tr>
- <tr>
- <td>meta.data_canal</td>
- <td>STRING/BYTES</td>
- <td>Data for rows in `canal-json` format only exists when the `source.multiple.enable` option is 'true'.</td>
- </tr>
- <tr>
- <td>meta.is_ddl</td>
- <td>BOOLEAN</td>
- <td>Whether the DDL statement.</td>
- </tr>
- <tr>
- <td>meta.ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>The current time when the row was received and processed.</td>
- </tr>
- <tr>
- <td>meta.sql_type</td>
- <td>MAP</td>
- <td>Mapping of sql_type table fields to java data type IDs.</td>
- </tr>
- <tr>
- <td>meta.oracle_type</td>
- <td>MAP</td>
- <td>Structure of the table.</td>
- </tr>
- <tr>
- <td>meta.pk_names</td>
- <td>ARRAY</td>
- <td>Primay key name of the table.</td>
- </tr>
- </tbody>
-</table>
+| **Key** | **DataType** | **Description** |
+| ------------------------- | ------------ | ---------- |
+| table_name| STRING NOT NULL| Name of the table that contain the row.|
+| schema_name| STRING NOT NULL| Name of the schema that contain the row.|
+| database_name| STRING NOT NULL| Name of the database that contain the row.|
+| op_ts| TIMESTAMP_LTZ(3) NOT NULL| It indicates the time that the change was made in the database. <br/>If the record is read from snapshot of the table instead of the change stream, the value is always 0.|
+| meta.table_name| STRING NOT NULL| Name of the table that contain the row.|
+| meta.schema_name| STRING NOT NULL| Name of the schema that contain the row.|
+| meta.database_name| STRING NOT NULL| Name of the database that contain the row.|
+| meta.op_ts| TIMESTAMP_LTZ(3) NOT NULL| It indicates the time that the change was made in the database. <br/>If the record is read from snapshot of the table instead of the change stream, the value is always 0.|
+| meta.op_type| STRING| Type of database operation, such as INSERT/DELETE, etc.|
+| meta.data_canal| STRING/BYTES| Data for rows in `canal-json` format only exists when the `source.multiple.enable` option is 'true'.|
+| meta.is_ddl| BOOLEAN| Whether the DDL statement.|
+| meta.ts| TIMESTAMP_LTZ(3) NOT NULL| The current time when the row was received and processed.|
+| meta.sql_type| MAP| Mapping of sql_type table fields to java data type IDs.|
+| meta.oracle_type| MAP| Structure of the table.|
+| meta.pk_names| ARRAY| Primay key name of the table.|
The extended CREATE TABLE example demonstrates the syntax for exposing these metadata fields:
```sql
@@ -570,7 +358,7 @@ Oracle Extract Node supports the whole database, multiple schemas, multiple tabl
config options:
-| Option | Required | Default | Type | Description |
+| **Option** | **Required** | **Default** | **Type** | **Description** |
| ---| ---| ---| ---| ---|
|source.multiple.enable|optional| false|String|Specify `'source.multiple.enable' = 'true'` to enable the whole database, multiple schemas, multiple tables migration function |
|schema-name|required|(none)|String| Schema name of the Oracle database to monitor. If you want to capture multiple schemas, you can use commas to separate them. For example: `'schema-name' = 'SCHEMA1,SCHEMA2'` |
@@ -596,109 +384,21 @@ CREATE TABLE node(
## Data Type Mapping
-<div class="wy-table-responsive">
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left">Oracle type<a href="https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Data-Types.html"></a></th>
- <th class="text-left">Flink SQL type</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 3
- </td>
- <td>TINYINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 5
- </td>
- <td>SMALLINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 10
- </td>
- <td>INT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 19
- </td>
- <td>BIGINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), 19 <= p - s <= 38 <br/>
- </td>
- <td>DECIMAL(p - s, 0)</td>
- </tr>
- <tr>
- <td>NUMBER(p, s > 0)
- </td>
- <td>DECIMAL(p, s)</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s > 38
- </td>
- <td>STRING</td>
- </tr>
- <tr>
- <td>
- FLOAT<br/>
- BINARY_FLOAT
- </td>
- <td>FLOAT</td>
- </tr>
- <tr>
- <td>
- DOUBLE PRECISION<br/>
- BINARY_DOUBLE
- </td>
- <td>DOUBLE</td>
- </tr>
- <tr>
- <td>NUMBER(1)</td>
- <td>BOOLEAN</td>
- </tr>
- <tr>
- <td>
- DATE<br/>
- TIMESTAMP [(p)]
- </td>
- <td>TIMESTAMP [(p)] [WITHOUT TIMEZONE]</td>
- </tr>
- <tr>
- <td>TIMESTAMP [(p)] WITH TIME ZONE</td>
- <td>TIMESTAMP [(p)] WITH TIME ZONE</td>
- </tr>
- <tr>
- <td>TIMESTAMP [(p)] WITH LOCAL TIME ZONE</td>
- <td>TIMESTAMP_LTZ [(p)]</td>
- </tr>
- <tr>
- <td>
- CHAR(n)<br/>
- NCHAR(n)<br/>
- NVARCHAR2(n)<br/>
- VARCHAR(n)<br/>
- VARCHAR2(n)<br/>
- CLOB<br/>
- NCLOB<br/>
- XMLType
- </td>
- <td>STRING</td>
- </tr>
- <tr>
- <td>BLOB<br/>
- ROWID
- </td>
- <td>BYTES</td>
- </tr>
- <tr>
- <td>
- INTERVAL DAY TO SECOND<br/>
- INTERVAL YEAR TO MONTH
- </td>
- <td>BIGINT</td>
- </tr>
- </tbody>
-</table>
-</div>
\ No newline at end of file
+| **[Oracle type](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Data-Types.html)** | **Flink SQL type**|
+| ---| ---|
+| NUMBER(p, s <= 0), p - s < 3 | TINYINT|
+| NUMBER(p, s <= 0), p - s < 5 | SMALLINT|
+| NUMBER(p, s <= 0), p - s < 10 | INT|
+| NUMBER(p, s <= 0), p - s < 19 | BIGINT|
+| NUMBER(p, s <= 0), 19 <= p - s <= 38 <br/> | DECIMAL(p - s, 0)|
+| NUMBER(p, s > 0) | DECIMAL(p, s)|
+| NUMBER(p, s <= 0), p - s > 38 | STRING|
+| FLOAT<br/> BINARY_FLOAT | FLOAT|
+| DOUBLE PRECISION<br/> BINARY_DOUBLE | DOUBLE|
+| NUMBER(1)| BOOLEAN|
+| DATE<br/> TIMESTAMP [(p)] | TIMESTAMP [(p)] [WITHOUT TIMEZONE]|
+| TIMESTAMP [(p)] WITH TIME ZONE| TIMESTAMP [(p)] WITH TIME ZONE|
+| TIMESTAMP [(p)] WITH LOCAL TIME ZONE| TIMESTAMP_LTZ [(p)]|
+| CHAR(n)<br/> NCHAR(n)<br/> NVARCHAR2(n)<br/> VARCHAR(n)<br/> VARCHAR2(n)<br/> CLOB<br/> NCLOB<br/> XMLType | STRING|
+| BLOB<br/> ROWID | BYTES|
+| INTERVAL DAY TO SECOND<br/> INTERVAL YEAR TO MONTH | BIGINT|
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/oracle-cdc.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/oracle-cdc.md
index d3c2a1d2ed..e5dd8f0fae 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/oracle-cdc.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/oracle-cdc.md
@@ -236,173 +236,29 @@ TODO: 将在未来支持此功能。
## Oracle Extact 节点参数
-<div class="highlight">
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left" style={{width: '25%'}}>参数</th>
- <th class="text-left" style={{width: '8%'}}>是否必须</th>
- <th class="text-left" style={{width: '7%'}}>默认值</th>
- <th class="text-left" style={{width: '10%'}}>数据类型</th>
- <th class="text-left" style={{width: '50%'}}>描述</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>connector</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>指定要使用的连接器，这里应该是 <code>'oracle-cdc-inlong'</code>。</td>
- </tr>
- <tr>
- <td>hostname</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>Oracle 数据库服务器的 IP 地址或主机名。</td>
- </tr>
- <tr>
- <td>username</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>连接到 Oracle 数据库服务器时要使用的 Oracle 数据库的名称。</td>
- </tr>
- <tr>
- <td>password</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>连接到 Oracle 数据库服务器时使用的密码。</td>
- </tr>
- <tr>
- <td>database-name</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>要监视的 Oracle 服务器的数据库名称。</td>
- </tr>
- <tr>
- <td>schema-name</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>要监视的 Oracle 数据库的 Schema 名称。</td>
- </tr>
- <tr>
- <td>table-name</td>
- <td>必选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>要监视的 Oracle 数据库的表名。格式为<i><schema_name>.<table_name></i></td>
- </tr>
- <tr>
- <td>port</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>1521</td>
- <td>Integer</td>
- <td>Oracle 数据库服务器的整数端口号。</td>
- </tr>
- <tr>
- <td>scan.startup.mode</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>initial</td>
- <td>String</td>
- <td>
-Oracle CDC 消费者的可选启动模式，有效枚举为"initial"
- 和"latest-offset"。
- 请参阅<a href="#startup-reading-position">启动阅读位置</a>部分了解更多详细信息。</td>
- </tr>
- <tr>
- <td>debezium.*</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>将 Debezium 的属性整合到用于从 Oracle 服务器捕获数据更改的 Debezium Embedded Engine。
- 例如：<code>'debezium.snapshot.mode' = 'never'</code>。
- 详细了解 <a href="https://debezium.io/documentation/reference/1.5/connectors/oracle.html#oracle-connector-properties">Debezium 的 Oracle 连接器属性</a></td>
- </tr>
- <tr>
- <td>inlong.metric.labels</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>(none)</td>
- <td>String</td>
- <td>inlong metric 的标签值，该值的构成为groupId=[groupId]&streamId=[streamId]&nodeId=[nodeId]。</td>
- </tr>
- <tr>
- <td>source.multiple.enable</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>false</td>
- <td>Boolean</td>
- <td>是否开启多模式、表同步功能，如果为 'true'，Oracle Extract Node 则将表的物理字段压缩成 'canal-json' 格式的特殊元字段 'data_canal'。</td>
- </tr>
- <tr>
- <td>scan.incremental.snapshot.enabled</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>true</td>
- <td>Boolean</td>
- <td>增量快照是一种读取表快照的新机制。与旧的快照机制相比，增量快照具有许多优点，包括：（1）在快照读取期间 Source 可以是并行的，（2）Source 可以在快照读取过程中执行 Chunk 粒度中的检查点，（3）Source 不需要在快照读取之前获取 ROW SHARE MODE 锁。</td>
- </tr>
- <tr>
- <td>scan.incremental.snapshot.chunk.size</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>8096</td>
- <td>Integer</td>
- <td>表快照的块大小（行数），读取表的快照时，表的快照被分成多个块。</td>
- </tr>
- <tr>
- <td>scan.snapshot.fetch.size</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>1024</td>
- <td>Integer</td>
- <td>读取表快照时每次轮询的最大获取大小。</td>
- </tr>
- <tr>
- <td>connect.max-retries</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>3</td>
- <td>Integer</td>
- <td>连接器应重试以建立 Oracle 数据库服务器连接的最大重试次数。</td>
- </tr>
- <tr>
- <td>chunk-meta.group.size</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>1000</td>
- <td>Integer</td>
- <td>Chunk meta 组大小，如果 meta 大小超过组大小，则 meta 将被分成多个组。</td>
- </tr>
- <tr>
- <td>connect.timeout</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>30s</td>
- <td>Duration</td>
- <td>连接器在尝试连接到 Oracle 数据库服务器后在超时之前应等待的最长时间。</td>
- </tr>
- <tr>
- <td>chunk-key.even-distribution.factor.lower-bound</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>0.05d</td>
- <td>Double</td>
- <td>Chunk Key 分布因子的下限。分布系数用于确定表格是否均匀分布。当数据分布均匀时，表块将均匀地使用计算优化，当数据分布不均匀时，将进行拆分查询。分布因子可以通过（MAX（id）-MIN（id）+1）/rowCount计算。</td>
- </tr>
- <tr>
- <td>chunk-key.even-distribution.factor.upper-bound</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>1000.0d</td>
- <td>Double</td>
- <td>Chunk Key 分布因子的上限。分布系数用于确定表格是否均匀分布。当数据分布均匀时，表块将均匀地使用计算优化，当数据分布不均匀时，将进行拆分查询。分布因子可以通过（MAX（id）-MIN（id）+1）/rowCount计算。</td>
- </tr>
- <tr>
- <td>connection.pool.size</td>
- <td>可选</td>
- <td style={{wordWrap: 'break-word'}}>20</td>
- <td>Integer</td>
- <td>连接池大小。</td>
- </tr>
- </tbody>
-</table>
-</div>
+| **选项** | **是否必须** | **默认** | **类型** | **描述** |
+| ------------------------- | ------------ | ---------- | -------- | ------------------------------------------------------------ |
+| connector| 必选| (none)| String| 指定要使用的连接器，这里应该是 <code>'oracle-cdc-inlong'</code>。|
+| hostname| 必选| (none)| String| Oracle 数据库服务器的 IP 地址或主机名。|
+| username| 必选| (none)| String| 连接到 Oracle 数据库服务器时要使用的 Oracle 数据库的名称。|
+| password| 必选| (none)| String| 连接到 Oracle 数据库服务器时使用的密码。|
+| database-name| 必选| (none)| String| 要监视的 Oracle 服务器的数据库名称。|
+| schema-name| 必选| (none)| String| 要监视的 Oracle 数据库的 Schema 名称。|
+| table-name| 必选| (none)| String| 要监视的 Oracle 数据库的表名。格式为<i><schema_name>.<table_name></i>|
+| port| 可选| 1521| Integer| Oracle 数据库服务器的整数端口号。|
+| scan.startup.mode| 可选| initial| String|Oracle CDC 消费者的可选启动模式，有效枚举为"initial"和"latest-offset"。 请参阅<a href="#startup-reading-position">启动阅读位置</a>部分了解更多详细信息。|
+| debezium.*| 可选| (none)| String| 将 Debezium 的属性整合到用于从 Oracle 服务器捕获数据更改的 Debezium Embedded Engine。 例如：<code>'debezium.snapshot.mode' = 'never'</code>。 详细了解 <a href="https://debezium.io/documentation/reference/1.5/connectors/oracle.html#oracle-connector-properties">Debezium 的 Oracle 连接器属性</a>|
+| inlong.metric.labels| 可选| (none)| String| inlong metric 的标签值，该值的构成为groupId=[groupId]&streamId=[streamId]&nodeId=[nodeId]。|
+| source.multiple.enable| 可选| false| Boolean| 是否开启多模式、表同步功能，如果为 'true'，Oracle Extract Node 则将表的物理字段压缩成 'canal-json' 格式的特殊元字段 'data_canal'。|
+| scan.incremental.snapshot.enabled| 可选| true| Boolean| 增量快照是一种读取表快照的新机制。与旧的快照机制相比，增量快照具有许多优点，包括：（1）在快照读取期间 Source 可以是并行的，（2）Source 可以在快照读取过程中执行 Chunk 粒度中的检查点，（3）Source 不需要在快照读取之前获取 ROW SHARE MODE 锁。|
+| scan.incremental.snapshot.chunk.size| 可选| 8096| Integer| 表快照的块大小（行数），读取表的快照时，表的快照被分成多个块。|
+| scan.snapshot.fetch.size| 可选| 1024| Integer| 读取表快照时每次轮询的最大获取大小。|
+| connect.max-retries| 可选| 3| Integer| 连接器应重试以建立 Oracle 数据库服务器连接的最大重试次数。|
+| chunk-meta.group.size| 可选| 1000| Integer| Chunk meta 组大小，如果 meta 大小超过组大小，则 meta 将被分成多个组。|
+| connect.timeout| 可选| 30s| Duration| 连接器在尝试连接到 Oracle 数据库服务器后在超时之前应等待的最长时间。|
+| chunk-key.even-distribution.factor.lower-bound| 可选| 0.05d| Double| Chunk Key 分布因子的下限。分布系数用于确定表格是否均匀分布。当数据分布均匀时，表块将均匀地使用计算优化，当数据分布不均匀时，将进行拆分查询。分布因子可以通过（MAX（id）-MIN（id）+1）/rowCount计算。|
+| chunk-key.even-distribution.factor.upper-bound| 可选| 1000.0d| Double| Chunk Key 分布因子的上限。分布系数用于确定表格是否均匀分布。当数据分布均匀时，表块将均匀地使用计算优化，当数据分布不均匀时，将进行拆分查询。分布因子可以通过（MAX（id）-MIN（id）+1）/rowCount计算。|
+| connection.pool.size| 可选| 20| Integer| 连接池大小。|
## 局限性
@@ -420,92 +276,24 @@ restart-strategy.fixed-delay.attempts: 2147483647
以下格式元数据可以作为表定义中的只读 (VIRTUAL) 列公开。
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left" style={{width: '15%'}}>字段名称</th>
- <th class="text-left" style={{width: '30%'}}>数据类型</th>
- <th class="text-left" style={{width: '55%'}}>描述</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>table_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的表名。</td>
- </tr>
- <tr>
- <td>schema_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的模式名称。</td>
- </tr>
- <tr>
- <td>database_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的数据库名称。</td>
- </tr>
- <tr>
- <td>op_ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>它指示在数据库中进行更改的时间。<br/>如果记录从表的快照而不是change流中读取，则该值始终为0。</td>
- </tr>
- <tr>
- <td>meta.table_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的表名。</td>
- </tr>
- <tr>
- <td>meta.schema_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的模式名称。</td>
- </tr>
- <tr>
- <td>meta.database_name</td>
- <td>STRING NOT NULL</td>
- <td>该行所属的数据库名称。</td>
- </tr>
- <tr>
- <td>meta.op_ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>它指示在数据库中进行更改的时间。<br/>如果记录从表的快照而不是change流中读取，则该值始终为0。</td>
- </tr>
- <tr>
- <td>meta.op_type</td>
- <td>STRING</td>
- <td>数据库操作的类型，如 INSERT/DELETE 等。</td>
- </tr>
- <tr>
- <td>meta.data_canal</td>
- <td>STRING/BYTES</td>
- <td>`canal-json` 格式化的行的数据只有在 `source.multiple.enable` 选项为 'true' 时才存在。</td>
- </tr>
- <tr>
- <td>meta.is_ddl</td>
- <td>BOOLEAN</td>
- <td>是否是 DDL 语句。</td>
- </tr>
- <tr>
- <td>meta.ts</td>
- <td>TIMESTAMP_LTZ(3) NOT NULL</td>
- <td>接收和处理行的当前时间。</td>
- </tr>
- <tr>
- <td>meta.sql_type</td>
- <td>MAP</td>
- <td>将 Sql_type 表字段映射到 Java 数据类型 Id。</td>
- </tr>
- <tr>
- <td>meta.oracle_type</td>
- <td>MAP</td>
- <td>表的结构。</td>
- </tr>
- <tr>
- <td>meta.pk_names</td>
- <td>ARRAY</td>
- <td>表的主键名称。</td>
- </tr>
- </tbody>
-</table>
+
+| **字段名称** | **数据类型** | **描述** |
+| ------------------------- | ------------ | ---------- |
+| table_name| STRING NOT NULL| 该行所属的表名。|
+| schema_name| STRING NOT NULL| 该行所属的模式名称。|
+| database_name| STRING NOT NULL| 该行所属的数据库名称。|
+| op_ts| TIMESTAMP_LTZ(3) NOT NULL| 它指示在数据库中进行更改的时间。<br/>如果记录从表的快照而不是change流中读取，则该值始终为0。|
+| meta.table_name| STRING NOT NULL| 该行所属的表名。|
+| meta.schema_name| STRING NOT NULL| 该行所属的模式名称。|
+| meta.database_name| STRING NOT NULL| 该行所属的数据库名称。|
+| meta.op_ts| TIMESTAMP_LTZ(3) NOT NULL| 它指示在数据库中进行更改的时间。<br/>如果记录从表的快照而不是change流中读取，则该值始终为0。|
+| meta.op_type| STRING| 数据库操作的类型，如 INSERT/DELETE 等。|
+| meta.data_canal| STRING/BYTES| `canal-json` 格式化的行的数据只有在 `source.multiple.enable` 选项为 'true' 时才存在。|
+| meta.is_ddl| BOOLEAN| 是否是 DDL 语句。|
+| meta.ts| TIMESTAMP_LTZ(3) NOT NULL| 接收和处理行的当前时间。|
+| meta.sql_type| MAP| 将 Sql_type 表字段映射到 Java 数据类型 Id。|
+| meta.oracle_type| MAP| 表的结构。|
+| meta.pk_names| ARRAY| 表的主键名称。|
扩展的 CREATE TABLE 示例演示了公开这些元数据字段的语法：
@@ -572,7 +360,7 @@ Oracle Extract 节点支持整库、多模式、多表同步。开启该功能
配置参数：
-| 参数 | 是否必须 | 默认值 | 数据类型 | 描述 |
+| **参数** | **是否必须** | **默认值** | **数据类型** | **描述** |
| ---| ---| ---| ---| ---|
|source.multiple.enable|optional| false|String| 指定`'source.multiple.enable' = 'true'`参数开启整库、多模式、多表同步功能 |
|schema-name|required|(none)|String| 要监视的 Oracle 数据库的 Schema 名称。如果要捕获多个模式，可以使用逗号分割它们。例如：`'schema-name' = 'SCHEMA1,SCHEMA2'` |
@@ -598,109 +386,21 @@ CREATE TABLE node(
## 数据类型映射
-<div class="wy-table-responsive">
-<table class="colwidths-auto docutils">
- <thead>
- <tr>
- <th class="text-left">Oracle type<a href="https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Data-Types.html"></a></th>
- <th class="text-left">Flink SQL type</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 3
- </td>
- <td>TINYINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 5
- </td>
- <td>SMALLINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 10
- </td>
- <td>INT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s < 19
- </td>
- <td>BIGINT</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), 19 <= p - s <= 38 <br/>
- </td>
- <td>DECIMAL(p - s, 0)</td>
- </tr>
- <tr>
- <td>NUMBER(p, s > 0)
- </td>
- <td>DECIMAL(p, s)</td>
- </tr>
- <tr>
- <td>NUMBER(p, s <= 0), p - s > 38
- </td>
- <td>STRING</td>
- </tr>
- <tr>
- <td>
- FLOAT<br/>
- BINARY_FLOAT
- </td>
- <td>FLOAT</td>
- </tr>
- <tr>
- <td>
- DOUBLE PRECISION<br/>
- BINARY_DOUBLE
- </td>
- <td>DOUBLE</td>
- </tr>
- <tr>
- <td>NUMBER(1)</td>
- <td>BOOLEAN</td>
- </tr>
- <tr>
- <td>
- DATE<br/>
- TIMESTAMP [(p)]
- </td>
- <td>TIMESTAMP [(p)] [WITHOUT TIMEZONE]</td>
- </tr>
- <tr>
- <td>TIMESTAMP [(p)] WITH TIME ZONE</td>
- <td>TIMESTAMP [(p)] WITH TIME ZONE</td>
- </tr>
- <tr>
- <td>TIMESTAMP [(p)] WITH LOCAL TIME ZONE</td>
- <td>TIMESTAMP_LTZ [(p)]</td>
- </tr>
- <tr>
- <td>
- CHAR(n)<br/>
- NCHAR(n)<br/>
- NVARCHAR2(n)<br/>
- VARCHAR(n)<br/>
- VARCHAR2(n)<br/>
- CLOB<br/>
- NCLOB<br/>
- XMLType
- </td>
- <td>STRING</td>
- </tr>
- <tr>
- <td>BLOB<br/>
- ROWID
- </td>
- <td>BYTES</td>
- </tr>
- <tr>
- <td>
- INTERVAL DAY TO SECOND<br/>
- INTERVAL YEAR TO MONTH
- </td>
- <td>BIGINT</td>
- </tr>
- </tbody>
-</table>
-</div>
\ No newline at end of file
+| **[Oracle type](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Data-Types.html)** | **Flink SQL type**|
+| ---| ---|
+| NUMBER(p, s <= 0), p - s < 3 | TINYINT|
+| NUMBER(p, s <= 0), p - s < 5 | SMALLINT|
+| NUMBER(p, s <= 0), p - s < 10 | INT|
+| NUMBER(p, s <= 0), p - s < 19 | BIGINT|
+| NUMBER(p, s <= 0), 19 <= p - s <= 38 <br/> | DECIMAL(p - s, 0)|
+| NUMBER(p, s > 0) | DECIMAL(p, s)|
+| NUMBER(p, s <= 0), p - s > 38 | STRING|
+| FLOAT<br/> BINARY_FLOAT | FLOAT|
+| DOUBLE PRECISION<br/> BINARY_DOUBLE | DOUBLE|
+| NUMBER(1)| BOOLEAN|
+| DATE<br/> TIMESTAMP [(p)] | TIMESTAMP [(p)] [WITHOUT TIMEZONE]|
+| TIMESTAMP [(p)] WITH TIME ZONE| TIMESTAMP [(p)] WITH TIME ZONE|
+| TIMESTAMP [(p)] WITH LOCAL TIME ZONE| TIMESTAMP_LTZ [(p)]|
+| CHAR(n)<br/> NCHAR(n)<br/> NVARCHAR2(n)<br/> VARCHAR(n)<br/> VARCHAR2(n)<br/> CLOB<br/> NCLOB<br/> XMLType | STRING|
+| BLOB<br/> ROWID | BYTES|
+| INTERVAL DAY TO SECOND<br/> INTERVAL YEAR TO MONTH | BIGINT|
\ No newline at end of file