You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zhangchengming601 (via GitHub)" <gi...@apache.org> on 2023/05/08 09:28:21 UTC
[GitHub] [incubator-seatunnel] zhangchengming601 opened a new pull request, #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
zhangchengming601 opened a new pull request, #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
## Check list
* [ ] Code changed are covered with tests, or it does not need tests for reason:
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
* [ ] If you are contributing the connector code, please check that the following files are updated:
1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/incubator-seatunnel/tree/dev/docs/en/connector-v2)
2. Update [plugin-mapping.properties](https://github.com/apache/incubator-seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
3. Update the pom file of [seatunnel-dist](https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-dist/pom.xml)
* [ ] Update the [`release-note`](https://github.com/apache/incubator-seatunnel/blob/dev/release-note.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322303058
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,159 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc.
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|---------------------------------|-----------------------------------------|------------------------------------------------------------------------------------|
+| redshift | Different dependency version has different driver class. | com.amazon.redshift.jdbc.Driver | jdbc:redshift://localhost:5439/database | [Download](https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example Redshift datasource: cp RedshiftJDBC42-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| Redshift Data type | Seatunnel Data type |
+|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| SMALLINT<br />INT2 | SHORT |
+| INTEGER<br />INT<br />INT4 | INT |
+| BIGINT<br />INT8<br />OID | LONG |
+| DECIMAL<br />NUMERIC | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) |
+| REAL<br />FLOAT4 | FLOAT |
+| DOUBLE_PRECISION<br />FLOAT8<br />FLOAT | DOUBLE |
+| BOOLEAN<br />BOOL | BOOLEAN |
+| CHAR<br />CHARACTER<br />NCHAR<br />BPCHAR<br />VARCHAR<br />CHARACTER_VARYING<br />NVARCHAR<br />TEXT<br />SUPER | STRING |
+| DATE | LOCALDATE |
+| TIME<br />TIME_WITH_TIME_ZONE<br />TIMETZ | LOCALTIME |
+| TIMESTAMP<br />TIMESTAMP_WITH_OUT_TIME_ZONE<br />TIMESTAMPTZ | LOCALDATETIME |
+| GEOMETRY | Not supported yet |
+
+## Options
Review Comment:
Source Options
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322302948
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,159 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc.
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|---------------------------------|-----------------------------------------|------------------------------------------------------------------------------------|
+| redshift | Different dependency version has different driver class. | com.amazon.redshift.jdbc.Driver | jdbc:redshift://localhost:5439/database | [Download](https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example Redshift datasource: cp RedshiftJDBC42-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| Redshift Data type | Seatunnel Data type |
+|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| SMALLINT<br />INT2 | SHORT |
+| INTEGER<br />INT<br />INT4 | INT |
+| BIGINT<br />INT8<br />OID | LONG |
+| DECIMAL<br />NUMERIC | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) |
+| REAL<br />FLOAT4 | FLOAT |
+| DOUBLE_PRECISION<br />FLOAT8<br />FLOAT | DOUBLE |
+| BOOLEAN<br />BOOL | BOOLEAN |
+| CHAR<br />CHARACTER<br />NCHAR<br />BPCHAR<br />VARCHAR<br />CHARACTER_VARYING<br />NVARCHAR<br />TEXT<br />SUPER | STRING |
+| DATE | LOCALDATE |
+| TIME<br />TIME_WITH_TIME_ZONE<br />TIMETZ | LOCALTIME |
+| TIMESTAMP<br />TIMESTAMP_WITH_OUT_TIME_ZONE<br />TIMESTAMPTZ | LOCALDATETIME |
+| GEOMETRY | Not supported yet |
+
+## Options
+
+| name | type | required | default | description |
+|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:redshift://localhost:5439/database |
+| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use Redshift the value is `com.amazon.redshift.jdbc.Driver`. |
+| user | String | No | - | Connection instance user name |
+| password | String | No | - | Connection instance password |
+| query | String | Yes | - | Query statement |
+| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
+| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type,Only support numeric type primary key, and only can config one column. |
+| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
+| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
+| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
+| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
+| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
+
+## tips
+
+> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
+
+## Task Example
+
+### simple:
+
+> This example defines a SeaTunnel synchronization task that automatically generates data through FakeSource and sends it to JDBC Sink. FakeSource generates a total of 16 rows of data (row.num=16), with each row having two fields, name (string type) and age (int type). The final target table is test_table will also be 16 rows of data in the table. Before run this job, you need create database mydatabase and table test_table in your Redshift. And if you have not yet installed and deployed SeaTunnel, you need to follow the instructions in [Install SeaTunnel](https://github.com/apache/incubator-seatunnel/blob/d013b5d2ac79f567077b62ceb4d247abf805ffdf/docs/en/start-v2/locally/deployment.md) to install and deploy SeaTunnel. And then follow the instructions in [Quick Start With SeaTunnel Engine](https://github.com/apache/incubator-seatunnel/blob/d013b5d2ac79f567077b62ceb4d247abf805ffdf/docs/en/start-v2/locally/quick-start-seatunnel-engine.md) to run this job.
+
+```
+# Defining the runtime environment
+env {
+ # You can set flink configuration here
+ execution.parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ # This is a example source plugin **only for test and demonstrate the feature source plugin**
+ FakeSource {
+ parallelism = 1
+ result_table_name = "fake"
+ row.num = 16
+ schema = {
+ fields {
+ name = "string"
+ age = "int"
Review Comment:
ditto
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,159 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc.
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|---------------------------------|-----------------------------------------|------------------------------------------------------------------------------------|
+| redshift | Different dependency version has different driver class. | com.amazon.redshift.jdbc.Driver | jdbc:redshift://localhost:5439/database | [Download](https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example Redshift datasource: cp RedshiftJDBC42-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| Redshift Data type | Seatunnel Data type |
+|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| SMALLINT<br />INT2 | SHORT |
+| INTEGER<br />INT<br />INT4 | INT |
+| BIGINT<br />INT8<br />OID | LONG |
+| DECIMAL<br />NUMERIC | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) |
+| REAL<br />FLOAT4 | FLOAT |
+| DOUBLE_PRECISION<br />FLOAT8<br />FLOAT | DOUBLE |
+| BOOLEAN<br />BOOL | BOOLEAN |
+| CHAR<br />CHARACTER<br />NCHAR<br />BPCHAR<br />VARCHAR<br />CHARACTER_VARYING<br />NVARCHAR<br />TEXT<br />SUPER | STRING |
+| DATE | LOCALDATE |
+| TIME<br />TIME_WITH_TIME_ZONE<br />TIMETZ | LOCALTIME |
+| TIMESTAMP<br />TIMESTAMP_WITH_OUT_TIME_ZONE<br />TIMESTAMPTZ | LOCALDATETIME |
+| GEOMETRY | Not supported yet |
+
+## Options
+
+| name | type | required | default | description |
+|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322302866
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|----------------------------------------------|---------------------------------|-----------------------------------------------------------------------------------|
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
+
+| name | type | required | default | Description |
+|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:sqlserver://localhost:1433 |
+| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use SQLserver the value is `com.microsoft.sqlserver.jdbc.SQLServerDriver`. |
+| user | String | No | - | Connection instance user name |
+| password | String | No | - | Connection instance password |
+| query | String | Yes | - | Query statement |
+| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
+| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type. |
+| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
+| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
+| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
+| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
+| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
+
+## tips
+
+> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
+
+## Task Example
+
+### simple:
+
+> This example defines a SeaTunnel synchronization task that automatically generates data through FakeSource and sends it to JDBC Sink. FakeSource generates a total of 16 rows of data (row.num=16), with each row having two fields, name (string type) and age (int type). The final target table is test_table will also be 16 rows of data in the table. Before run this job, you need create database test and table test_table in your SQL Server. And if you have not yet installed and deployed SeaTunnel, you need to follow the instructions in [Install SeaTunnel](../../start-v2/locally/deployment.md) to install and deploy SeaTunnel. And then follow the instructions in [Quick Start With SeaTunnel Engine](../../start-v2/locally/quick-start-seatunnel-engine.md) to run this job.
+
+```
+# Defining the runtime environment
+env {
+ # You can set flink configuration here
+ execution.parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ # This is a example source plugin **only for test and demonstrate the feature source plugin**
+ FakeSource {
+ parallelism = 1
+ result_table_name = "fake"
+ row.num = 16
+ schema = {
+ fields {
+ name = "string"
+ age = "int"
+ }
+ }
+ }
+ # If you would like to get more information about how to configure seatunnel and see full list of source plugins,
+ # please go to https://seatunnel.apache.org/docs/category/source-v2
+}
+
+transform {
+ # If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
+ # please go to https://seatunnel.apache.org/docs/category/transform-v2
+}
+
+sink {
+ jdbc {
+ url = "jdbc:sqlserver://localhost:1433;databaseName=mydatabase"
+ driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
+ user = "root"
+ password = "123456"
+ query = "insert into test_table(name,age) values(?,?)"
+ }
+ # If you would like to get more information about how to configure seatunnel and see full list of sink plugins,
+ # please go to https://seatunnel.apache.org/docs/category/sink-v2
+}
+```
+
+### Exactly-once :
+
+> For accurate write scene we guarantee accurate once
+
+```
+jdbc {
+ url = "jdbc:sqlserver://localhost:1433;databaseName=testdb"
+ driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
+
+ max_retries = 0
+ user = "root"
+ password = "123456"
+ query = "insert into test_table(name,age) values(?,?)"
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on code in PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#discussion_r1196443372
##########
docs/en/connector-v2/source/SQLServer.md:
##########
@@ -0,0 +1,154 @@
+# SQL Server
+
+> JDBC SQL Server Source Connector
+
+## Support those engines
+
+> Spark <br>
+> Flink <br>
Review Comment:
```suggestion
> Flink <br/>
```
##########
docs/en/connector-v2/source/SQLServer.md:
##########
@@ -0,0 +1,154 @@
+# SQL Server
+
+> JDBC SQL Server Source Connector
+
+## Support those engines
+
+> Spark <br>
Review Comment:
```suggestion
> Spark <br/>
```
##########
docs/en/connector-v2/source/SQLServer.md:
##########
@@ -0,0 +1,154 @@
+# SQL Server
+
+> JDBC SQL Server Source Connector
+
+## Support those engines
+
+> Spark <br>
+> Flink <br>
+> Seatunnel Zeta <br>
Review Comment:
```suggestion
> Seatunnel Zeta <br/>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on code in PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#discussion_r1196706978
##########
docs/en/connector-v2/source/SqlServer-CDC.md:
##########
@@ -167,6 +167,35 @@ source {
Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details.
+## notice
Review Comment:
I think update this doc to the new format is better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#issuecomment-1733248704
https://github.com/apache/seatunnel/pull/4721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#issuecomment-1551672151
fix ci error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322303436
##########
docs/en/connector-v2/source/redshift.md:
##########
@@ -0,0 +1,147 @@
+# Redshift
+
+> JDBC Redshift Source Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [column projection](../../concept/connector-v2-features.md)
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [x] [support user-defined split](../../concept/connector-v2-features.md)
+
+> supports query SQL and can achieve projection effect.
+
+## Description
+
+Read external data source data through JDBC.
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|---------------------------------|-----------------------------------------|------------------------------------------------------------------------------------|
+| redshift | Different dependency version has different driver class. | com.amazon.redshift.jdbc.Driver | jdbc:redshift://localhost:5439/database | [Download](https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example Redshift datasource: cp RedshiftJDBC42-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| Redshift Data type | Seatunnel Data type |
+|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| SMALLINT<br />INT2 | SHORT |
+| INTEGER<br />INT<br />INT4 | INT |
+| BIGINT<br />INT8<br />OID | LONG |
+| DECIMAL<br />NUMERIC | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) |
+| REAL<br />FLOAT4 | FLOAT |
+| DOUBLE_PRECISION<br />FLOAT8<br />FLOAT | DOUBLE |
+| BOOLEAN<br />BOOL | BOOLEAN |
+| CHAR<br />CHARACTER<br />NCHAR<br />BPCHAR<br />VARCHAR<br />CHARACTER_VARYING<br />NVARCHAR<br />TEXT<br />SUPER | STRING |
+| DATE | LOCALDATE |
+| TIME<br />TIME_WITH_TIME_ZONE<br />TIMETZ | LOCALTIME |
+| TIMESTAMP<br />TIMESTAMP_WITH_OUT_TIME_ZONE<br />TIMESTAMPTZ | LOCALDATETIME |
+| GEOMETRY | Not supported yet |
+
+## Options
Review Comment:
Source Options
##########
docs/en/connector-v2/source/redshift.md:
##########
@@ -0,0 +1,147 @@
+# Redshift
+
+> JDBC Redshift Source Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [column projection](../../concept/connector-v2-features.md)
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [x] [support user-defined split](../../concept/connector-v2-features.md)
+
+> supports query SQL and can achieve projection effect.
+
+## Description
+
+Read external data source data through JDBC.
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|---------------------------------|-----------------------------------------|------------------------------------------------------------------------------------|
+| redshift | Different dependency version has different driver class. | com.amazon.redshift.jdbc.Driver | jdbc:redshift://localhost:5439/database | [Download](https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example Redshift datasource: cp RedshiftJDBC42-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| Redshift Data type | Seatunnel Data type |
+|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| SMALLINT<br />INT2 | SHORT |
+| INTEGER<br />INT<br />INT4 | INT |
+| BIGINT<br />INT8<br />OID | LONG |
+| DECIMAL<br />NUMERIC | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) |
+| REAL<br />FLOAT4 | FLOAT |
+| DOUBLE_PRECISION<br />FLOAT8<br />FLOAT | DOUBLE |
+| BOOLEAN<br />BOOL | BOOLEAN |
+| CHAR<br />CHARACTER<br />NCHAR<br />BPCHAR<br />VARCHAR<br />CHARACTER_VARYING<br />NVARCHAR<br />TEXT<br />SUPER | STRING |
+| DATE | LOCALDATE |
+| TIME<br />TIME_WITH_TIME_ZONE<br />TIMETZ | LOCALTIME |
+| TIMESTAMP<br />TIMESTAMP_WITH_OUT_TIME_ZONE<br />TIMESTAMPTZ | LOCALDATETIME |
+| GEOMETRY | Not supported yet |
+
+## Options
+
+| name | type | required | default | description |
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#issuecomment-1733248390
> @zhilinli123 Please fix ci error.
To replace the current pr links to https://github.com/apache/seatunnel/pull/5498
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322302722
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|----------------------------------------------|---------------------------------|-----------------------------------------------------------------------------------|
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
+
+| name | type | required | default | Description |
+|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:sqlserver://localhost:1433 |
+| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use SQLserver the value is `com.microsoft.sqlserver.jdbc.SQLServerDriver`. |
+| user | String | No | - | Connection instance user name |
+| password | String | No | - | Connection instance password |
+| query | String | Yes | - | Query statement |
+| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
+| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type. |
+| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
+| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
+| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
+| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
+| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
+
+## tips
+
+> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
+
+## Task Example
+
+### simple:
+
+> This example defines a SeaTunnel synchronization task that automatically generates data through FakeSource and sends it to JDBC Sink. FakeSource generates a total of 16 rows of data (row.num=16), with each row having two fields, name (string type) and age (int type). The final target table is test_table will also be 16 rows of data in the table. Before run this job, you need create database test and table test_table in your SQL Server. And if you have not yet installed and deployed SeaTunnel, you need to follow the instructions in [Install SeaTunnel](../../start-v2/locally/deployment.md) to install and deploy SeaTunnel. And then follow the instructions in [Quick Start With SeaTunnel Engine](../../start-v2/locally/quick-start-seatunnel-engine.md) to run this job.
+
+```
+# Defining the runtime environment
+env {
+ # You can set flink configuration here
+ execution.parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ # This is a example source plugin **only for test and demonstrate the feature source plugin**
+ FakeSource {
+ parallelism = 1
+ result_table_name = "fake"
+ row.num = 16
+ schema = {
+ fields {
+ name = "string"
+ age = "int"
Review Comment:
Add SqlSerber so type testing is sure to work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#discussion_r1187385579
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource list
+
+| datasource | supported versions | driver | url | maven |
+| ---------- | -------------------------------------------------------- | -------------------------------------------- | ------------------------------- | ------------------------------------------------------------ |
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
Review Comment:
Same as above
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,159 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
Review Comment:
Same as above
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource list
+
+| datasource | supported versions | driver | url | maven |
+| ---------- | -------------------------------------------------------- | -------------------------------------------- | ------------------------------- | ------------------------------------------------------------ |
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
+
+| name | type | required | default | Description |
+| ---------------------------- | ------ | -------- | --------------- | ------------------------------------------------------------ |
+| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:sqlserver://localhost:1433 |
+| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use SQLserver the value is `com.microsoft.sqlserver.jdbc.SQLServerDriver`. |
+| user | String | No | - | Connection instance user name |
+| password | String | No | - | Connection instance password |
+| query | String | Yes | - | Query statement |
+| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
+| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type. |
+| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
+| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
+| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
+| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
+| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
+
+## tips
+
+> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
+
+## Task Example
+
+### simple:
+
+> This example defines a SeaTunnel synchronization task that automatically generates data through FakeSource and sends it to JDBC Sink. FakeSource generates a total of 16 rows of data (row.num=16), with each row having two fields, name (string type) and age (int type). The final target table is test_table will also be 16 rows of data in the table. Before run this job, you need create database test and table test_table in your SQL Server. And if you have not yet installed and deployed SeaTunnel, you need to follow the instructions in [Install SeaTunnel](../../start-v2/locally/deployment.md) to install and deploy SeaTunnel. And then follow the instructions in [Quick Start With SeaTunnel Engine](../../start-v2/locally/quick-start-seatunnel-engine.md) to run this job.
+
+```
+# Defining the runtime environment
+env {
+ # You can set flink configuration here
+ execution.parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ # This is a example source plugin **only for test and demonstrate the feature source plugin**
Review Comment:
Same as above
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
Review Comment:
Please remove Support engines, connector-v2 always support all engine.
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource list
+
+| datasource | supported versions | driver | url | maven |
+| ---------- | -------------------------------------------------------- | -------------------------------------------- | ------------------------------- | ------------------------------------------------------------ |
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
+
+| name | type | required | default | Description |
+| ---------------------------- | ------ | -------- | --------------- | ------------------------------------------------------------ |
+| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:sqlserver://localhost:1433 |
+| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use SQLserver the value is `com.microsoft.sqlserver.jdbc.SQLServerDriver`. |
+| user | String | No | - | Connection instance user name |
+| password | String | No | - | Connection instance password |
+| query | String | Yes | - | Query statement |
+| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
+| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type. |
+| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
+| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
+| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
+| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
+| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
+
+## tips
+
+> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
+
+## Task Example
+
+### simple:
+
+> This example defines a SeaTunnel synchronization task that automatically generates data through FakeSource and sends it to JDBC Sink. FakeSource generates a total of 16 rows of data (row.num=16), with each row having two fields, name (string type) and age (int type). The final target table is test_table will also be 16 rows of data in the table. Before run this job, you need create database test and table test_table in your SQL Server. And if you have not yet installed and deployed SeaTunnel, you need to follow the instructions in [Install SeaTunnel](../../start-v2/locally/deployment.md) to install and deploy SeaTunnel. And then follow the instructions in [Quick Start With SeaTunnel Engine](../../start-v2/locally/quick-start-seatunnel-engine.md) to run this job.
+
+```
+# Defining the runtime environment
+env {
+ # You can set flink configuration here
Review Comment:
Remove unnecessary comment.
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
Review Comment:
Sink always support batch and stream
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource list
Review Comment:
It's weird, SQLServer sink only support SQLServer, why use `Supported DataSource list`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322302291
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|----------------------------------------------|---------------------------------|-----------------------------------------------------------------------------------|
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
+
+| name | type | required | default | Description |
+|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Review Comment:
| Name | Type | Required | Default | Description|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc][Sqlserver]Add notice into Sqlserver cdc doc [seatunnel]
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 closed pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
URL: https://github.com/apache/seatunnel/pull/4721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] EricJoy2048 closed pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 closed pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
URL: https://github.com/apache/seatunnel/pull/4721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] EricJoy2048 commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#issuecomment-1732876999
@zhilinli123 Please fix ci error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on code in PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#discussion_r1194806898
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,160 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource list
Review Comment:
```suggestion
## Supported DataSource Info
```
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,160 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
Review Comment:
```suggestion
```
##########
docs/en/connector-v2/source/redshift.md:
##########
@@ -0,0 +1,146 @@
+# Redshift
+
+> JDBC Redshift Source Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [column projection](../../concept/connector-v2-features.md)
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [x] [support user-defined split](../../concept/connector-v2-features.md)
+
+> supports query SQL and can achieve projection effect.
+
+## Description
+
+Read external data source data through JDBC.
+
+## Supported DataSource list
Review Comment:
```suggestion
## Supported DataSource Info
```
##########
docs/en/connector-v2/source/SQLServer.md:
##########
@@ -0,0 +1,154 @@
+# SQL Server
+
+> JDBC SQL Server Source Connector
+
+## Support those engines
+
+> Spark <br>
+> Flink <br>
+> Seatunnel Zeta <br>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [column projection](../../concept/connector-v2-features.md)
+
+supports query SQL and can achieve projection effect.
+
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [x] [support user-defined split](../../concept/connector-v2-features.md)
+
+## Description
+
+Read external data source data through JDBC.
+
+## Supported DataSource list
Review Comment:
```suggestion
## Supported DataSource Info
```
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,160 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [ ] [batch](../../concept/connector-v2-features.md)
Review Comment:
```suggestion
```
##########
docs/en/connector-v2/source/redshift.md:
##########
@@ -0,0 +1,146 @@
+# Redshift
+
+> JDBC Redshift Source Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
Review Comment:
```suggestion
- [ ] [stream](../../concept/connector-v2-features.md)
- [x] [exactly-once](../../concept/connector-v2-features.md)
```
##########
docs/en/connector-v2/sink/redshift.md:
##########
@@ -0,0 +1,160 @@
+# Redshift
+
+> JDBC Redshift sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [ ] [batch](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc.
+
+## Supported DataSource list
Review Comment:
```suggestion
## Supported DataSource Info
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhangchengming601 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhangchengming601 (via GitHub)" <gi...@apache.org>.
zhangchengming601 commented on code in PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#discussion_r1189333031
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
Review Comment:
I wrote it according to document standards
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#issuecomment-1538278537
Each doc have same problems, please fix it. Thansk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4721:
URL: https://github.com/apache/incubator-seatunnel/pull/4721#issuecomment-1538480751
Please close duplicate pull requests @zhangchengming601
I added some review comments in this community boss help to open up :https://github.com/apache/incubator-seatunnel/pull/4715
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #4721: [Doc][Sqlserver]Add notice into Sqlserver cdc doc
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4721:
URL: https://github.com/apache/seatunnel/pull/4721#discussion_r1322301967
##########
docs/en/connector-v2/sink/SQLServer.md:
##########
@@ -0,0 +1,159 @@
+# SQL Server
+
+> JDBC SQL Server Sink Connector
+
+## Support those engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key features
+
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [cdc](../../concept/connector-v2-features.md)
+
+> Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` for the database which is
+> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
+
+## Description
+
+Write data through jdbc. Support Batch mode and Streaming mode, support concurrent writing, support exactly-once
+semantics (using XA transaction guarantee).
+
+## Supported DataSource Info
+
+| datasource | supported versions | driver | url | maven |
+|------------|----------------------------------------------------------|----------------------------------------------|---------------------------------|-----------------------------------------------------------------------------------|
+| SQL Server | Different dependency version has different driver class. | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | [Download](https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc) |
+
+## Database dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example SQL Server datasource: cp mssql-jdbc-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+| SQLserver Data type | Seatunnel Data type |
+|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| BIT | BOOLEAN |
+| TINYINT<br/>SMALLINT | SHORT |
+| INTEGER | INT |
+| BIGINT | LONG |
+| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) |
+| REAL | FLOAT |
+| FLOAT | DOUBLE |
+| CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING |
+| DATE | LOCAL_DATE |
+| TIME | LOCAL_TIME |
+| DATETIME<br />DATETIME2<br />SMALLDATETIME<br />DATETIMEOFFSET | LOCAL_DATE_TIME |
+| TIMESTAMP<br />BINARY<br />VARBINARY<br />IMAGE<br />UNKNOWN | Not supported yet |
+
+## Options
Review Comment:
Sink Options
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org