You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by wa...@apache.org on 2024/01/15 12:34:41 UTC
(seatunnel) branch dev updated: [Feature][doc] Add V2 connector jdbc document parameter can speed up data import PR (#6176)
This is an automated email from the ASF dual-hosted git repository.
wanghailin pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 6a83169c54 [Feature][doc] Add V2 connector jdbc document parameter can speed up data import PR (#6176)
6a83169c54 is described below
commit 6a83169c5488f66a27b9203338100ab78b207779
Author: chenyunde <58...@users.noreply.github.com>
AuthorDate: Mon Jan 15 20:34:36 2024 +0800
[Feature][doc] Add V2 connector jdbc document parameter can speed up data import PR (#6176)
---
docs/en/connector-v2/sink/Jdbc.md | 19 ++++++++++++-------
docs/en/connector-v2/sink/Kingbase.md | 1 +
docs/en/connector-v2/sink/Mysql.md | 9 +++++----
docs/en/connector-v2/sink/OceanBase.md | 3 ++-
docs/en/connector-v2/sink/Oracle.md | 9 +++++----
docs/en/connector-v2/sink/PostgreSql.md | 1 +
docs/en/connector-v2/sink/Snowflake.md | 3 ++-
docs/en/connector-v2/sink/SqlServer.md | 1 +
docs/en/connector-v2/sink/Vertica.md | 1 +
9 files changed, 30 insertions(+), 17 deletions(-)
diff --git a/docs/en/connector-v2/sink/Jdbc.md b/docs/en/connector-v2/sink/Jdbc.md
index a9021b5f5e..4e4a8b704e 100644
--- a/docs/en/connector-v2/sink/Jdbc.md
+++ b/docs/en/connector-v2/sink/Jdbc.md
@@ -49,10 +49,11 @@ support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
| auto_commit | Boolean | No | true |
| field_ide | String | No | - |
| properties | Map | No | - |
-| common-options | | no | - |
-| schema_save_mode | Enum | no | CREATE_SCHEMA_WHEN_NOT_EXIST |
-| data_save_mode | Enum | no | APPEND_DATA |
-| custom_sql | String | no | - |
+| common-options | | No | - |
+| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST |
+| data_save_mode | Enum | No | APPEND_DATA |
+| custom_sql | String | No | - |
+| enable_upsert | Boolean | No | true |
### driver [string]
@@ -169,7 +170,7 @@ Additional connection configuration parameters,when properties and URL have the
Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details
-### schema_save_mode[Enum]
+### schema_save_mode [Enum]
Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side.
Option introduction:
@@ -177,7 +178,7 @@ Option introduction:
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
-### data_save_mode[Enum]
+### data_save_mode [Enum]
Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side.
Option introduction:
@@ -186,10 +187,14 @@ Option introduction:
`CUSTOM_PROCESSING`:User defined processing
`ERROR_WHEN_DATA_EXISTS`:When there is data, an error is reported
-### custom_sql[String]
+### custom_sql [String]
When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks.
+### enable_upsert [boolean]
+
+Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import
+
## tips
In the case of is_exactly_once = "true", Xa transactions are used. This requires database support, and some databases require some setup :
diff --git a/docs/en/connector-v2/sink/Kingbase.md b/docs/en/connector-v2/sink/Kingbase.md
index 6df11a1dcc..c2204d0209 100644
--- a/docs/en/connector-v2/sink/Kingbase.md
+++ b/docs/en/connector-v2/sink/Kingbase.md
@@ -74,6 +74,7 @@
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
### Tips
diff --git a/docs/en/connector-v2/sink/Mysql.md b/docs/en/connector-v2/sink/Mysql.md
index e5f750bcba..10dd1c526d 100644
--- a/docs/en/connector-v2/sink/Mysql.md
+++ b/docs/en/connector-v2/sink/Mysql.md
@@ -80,10 +80,11 @@ semantics (using XA transaction guarantee).
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| field_ide | String | No | - | Identify whether the field needs to be converted when synchronizing from the source to the sink. `ORIGINAL` indicates no conversion is needed;`UPPERCASE` indicates conversion to uppercase;`LOWERCASE` indicates conversion to lowercase. |
| properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. |
-| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
-| schema_save_mode | Enum | no | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. |
-| data_save_mode | Enum | no | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. |
-| custom_sql | String | no | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. |
+| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. |
+| data_save_mode | Enum | No | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. |
+| custom_sql | String | No | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task only has `insert`, setting this parameter to `false` can speed up data import |
### Tips
diff --git a/docs/en/connector-v2/sink/OceanBase.md b/docs/en/connector-v2/sink/OceanBase.md
index 15ec7abae5..ce60b0937d 100644
--- a/docs/en/connector-v2/sink/OceanBase.md
+++ b/docs/en/connector-v2/sink/OceanBase.md
@@ -87,7 +87,8 @@ Write data through jdbc. Support Batch mode and Streaming mode, support concurre
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. |
-| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
### Tips
diff --git a/docs/en/connector-v2/sink/Oracle.md b/docs/en/connector-v2/sink/Oracle.md
index 38edb5432d..e99b9ba89d 100644
--- a/docs/en/connector-v2/sink/Oracle.md
+++ b/docs/en/connector-v2/sink/Oracle.md
@@ -74,10 +74,11 @@ semantics (using XA transaction guarantee).
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. |
-| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
-| schema_save_mode | Enum | no | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. |
-| data_save_mode | Enum | no | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. |
-| custom_sql | String | no | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. |
+| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. |
+| data_save_mode | Enum | No | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. |
+| custom_sql | String | No | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
### Tips
diff --git a/docs/en/connector-v2/sink/PostgreSql.md b/docs/en/connector-v2/sink/PostgreSql.md
index 6164c54942..31f30b7346 100644
--- a/docs/en/connector-v2/sink/PostgreSql.md
+++ b/docs/en/connector-v2/sink/PostgreSql.md
@@ -87,6 +87,7 @@ semantics (using XA transaction guarantee).
| schema_save_mode | Enum | no | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. [...]
| data_save_mode | Enum | no | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. [...]
| custom_sql | String | no | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. [...]
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import [...]
### table [string]
diff --git a/docs/en/connector-v2/sink/Snowflake.md b/docs/en/connector-v2/sink/Snowflake.md
index f3b36c6134..62f9bd86ea 100644
--- a/docs/en/connector-v2/sink/Snowflake.md
+++ b/docs/en/connector-v2/sink/Snowflake.md
@@ -66,7 +66,8 @@ Write data through jdbc. Support Batch mode and Streaming mode, support concurre
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. |
-| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
## tips
diff --git a/docs/en/connector-v2/sink/SqlServer.md b/docs/en/connector-v2/sink/SqlServer.md
index 354641da1b..761af9c1ee 100644
--- a/docs/en/connector-v2/sink/SqlServer.md
+++ b/docs/en/connector-v2/sink/SqlServer.md
@@ -76,6 +76,7 @@ semantics (using XA transaction guarantee).
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
## tips
diff --git a/docs/en/connector-v2/sink/Vertica.md b/docs/en/connector-v2/sink/Vertica.md
index 8e7196d746..f8c6c4a746 100644
--- a/docs/en/connector-v2/sink/Vertica.md
+++ b/docs/en/connector-v2/sink/Vertica.md
@@ -76,6 +76,7 @@ semantics (using XA transaction guarantee).
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
| properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. |
| common-options | | no | - | Sink plugin common parameters, please refer to [Sink Common Options](common-options.md) for details |
+| enable_upsert | Boolean | No | true | Enable upsert by primary_keys exist, If the task has no key duplicate data, setting this parameter to `false` can speed up data import |
### Tips