You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/09/28 13:33:32 UTC
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #2832: [Feature][Connector-V2] add tdengine source
hailin0 commented on code in PR #2832:
URL: https://github.com/apache/incubator-seatunnel/pull/2832#discussion_r982402124
##########
docs/en/connector-v2/source/TDengine.md:
##########
@@ -0,0 +1,107 @@
+# TDengine
+
+> TDengine source connector
+
+## Description
+
+Read external data source data through TDengine.
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [ ] [schema projection](../../concept/connector-v2-features.md)
+
+supports query SQL and can achieve projection effect.
+
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [ ] [support user-defined split](../../concept/connector-v2-features.md)
+
+## Options
+
+| name | type | required | default value |
+|----------------------------|---------|----------|---------------|
+| url | string | yes | - |
+| username | string | yes | - |
+| password | string | yes | - |
+| database | string | yes | |
+| stable | string | yes | - |
+| partitions_num | int | no | - |
+| fields | config | no | - |
+| lower_bound | long | no | - |
+| upper_bound | long | no | - |
Review Comment:
check style
##########
docs/en/connector-v2/source/TDengine.md:
##########
@@ -0,0 +1,107 @@
+# TDengine
+
+> TDengine source connector
+
+## Description
+
+Read external data source data through TDengine.
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [ ] [schema projection](../../concept/connector-v2-features.md)
+
+supports query SQL and can achieve projection effect.
+
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [ ] [support user-defined split](../../concept/connector-v2-features.md)
+
+## Options
+
+| name | type | required | default value |
+|----------------------------|---------|----------|---------------|
+| url | string | yes | - |
+| username | string | yes | - |
+| password | string | yes | - |
+| database | string | yes | |
+| stable | string | yes | - |
+| partitions_num | int | no | - |
+| fields | config | no | - |
+| lower_bound | long | no | - |
+| upper_bound | long | no | - |
+
+### url [string]
+
+the url of the TDengine when you select the TDengine
+
+e.g.
+```
+jdbc:TAOS-RS://localhost:6041/
+```
+
+### fields [string]
+
+the fields of the TDengine when you select
+
+the field type is SeaTunnel field type `org.apache.seatunnel.api.table.type.SqlType`
+
+e.g.
+
+```
+ fields {
+ ts = "timestamp"
+ current = "float"
+ voltage = "int"
+ phase = "float"
+ location = "string"
+ groupid = "int"
+ }
+```
+
+### username [string]
+
+the username of the TDengine when you select
+
+### password [string]
+
+the password of the TDengine when you select
+
+### database [string]
+
+the database of the TDengine when you select
+
+### stable [string]
+
+the stable of the TDengine when you select
+
+### lower_bound [long]
+
+the lower_bound of the migration period
+
+### upper_bound [long]
+
+the upper_bound of the migration period
+
+### partitions_num [int]
+
+the partitions_num of the migration data
+
+
+```
+ split the time range into partitions_num parts
+ if numPartitions is 1, use the whole time range
+ if numPartitions < (upper_bound - lower_bound), use (upper_bound - lower_bound) partitions
+
+ eg: lower_bound = 1, upper_bound = 10, partitions_num = 2
+ sql = "select * from test"
+
+ split result
+
+ split 1: select * from test where (ts >= 1 and ts < 6)
+
+ split 2: select * from test where (ts >= 6 and ts < 11)
+
+```
+
Review Comment:
Add example?
reference
https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/connector-v2/source/IoTDB.md#examples
##########
seatunnel-connectors-v2-dist/pom.xml:
##########
@@ -0,0 +1,193 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
remove this module
##########
docs/en/connector-v2/source/TDengine.md:
##########
@@ -0,0 +1,107 @@
+# TDengine
+
+> TDengine source connector
+
+## Description
+
+Read external data source data through TDengine.
+
+## Key features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [ ] [schema projection](../../concept/connector-v2-features.md)
+
+supports query SQL and can achieve projection effect.
+
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [ ] [support user-defined split](../../concept/connector-v2-features.md)
+
+## Options
+
+| name | type | required | default value |
+|----------------------------|---------|----------|---------------|
+| url | string | yes | - |
+| username | string | yes | - |
+| password | string | yes | - |
+| database | string | yes | |
+| stable | string | yes | - |
+| partitions_num | int | no | - |
+| fields | config | no | - |
+| lower_bound | long | no | - |
+| upper_bound | long | no | - |
+
+### url [string]
+
+the url of the TDengine when you select the TDengine
+
+e.g.
+```
+jdbc:TAOS-RS://localhost:6041/
+```
+
+### fields [string]
+
+the fields of the TDengine when you select
+
+the field type is SeaTunnel field type `org.apache.seatunnel.api.table.type.SqlType`
+
+e.g.
+
+```
Review Comment:
```suggestion
```hocon
```
##########
seatunnel-connectors-v2/connector-tdengine/pom.xml:
##########
@@ -0,0 +1,46 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
Add this module into [here](https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-dist/pom.xml#L232)
##########
docs/en/connector-v2/source/TDengine.md:
##########
@@ -0,0 +1,107 @@
+# TDengine
Review Comment:
Add TDengine sink docs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org