You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zhilinli123 (via GitHub)" <gi...@apache.org> on 2023/09/18 14:12:39 UTC
[GitHub] [seatunnel] zhilinli123 opened a new pull request, #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
zhilinli123 opened a new pull request, #5255:
URL: https://github.com/apache/seatunnel/pull/5255
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
subTask:https://github.com/apache/seatunnel/issues/4572
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
## Check list
* [ ] Code changed are covered with tests, or it does not need tests for reason:
* [x] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If you are contributing the connector code, please check that the following files are updated:
1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/seatunnel/tree/dev/docs/en/connector-v2)
2. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
3. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
* [ ] Update the [`release-note`](https://github.com/apache/seatunnel/blob/dev/release-note.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363114667
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| rows | config | no | - | The row list of fake data output per degree of parallelism |
+| row.num | int | no | 5 | The total number of data generated per degree of parallelism |
+| split.num | int | no | 1 | the number of splits generated by the enumerator for each degree of parallelism |
+| split.read-interval | long | no | 1 | The interval(mills) between two split reads in a reader |
+| map.size | int | no | 5 | The size of `map` type that connector generated |
+| array.size | int | no | 5 | The size of `array` type that connector generated |
+| bytes.length | int | no | 5 | The length of `bytes` type that connector generated |
+| string.length | int | no | 5 | The length of `string` type that connector generated |
+| string.fake.mode | string | no | range | The fake mode of generating string data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `string.template` option |
+| string.template | list | no | - | The template list of string type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| tinyint.fake.mode | string | no | range | The fake mode of generating tinyint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `tinyint.template` option |
+| tinyint.min | tinyint | no | 0 | The min value of tinyint data that connector generated |
+| tinyint.max | tinyint | no | 127 | The max value of tinyint data that connector generated |
+| tinyint.template | list | no | - | The template list of tinyint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| smallint.fake.mode | string | no | range | The fake mode of generating smallint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `smallint.template` option |
+| smallint.min | smallint | no | 0 | The min value of smallint data that connector generated |
+| smallint.max | smallint | no | 32767 | The max value of smallint data that connector generated |
+| smallint.template | list | no | - | The template list of smallint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| int.fake.template | string | no | range | The fake mode of generating int data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `int.template` option |
+| int.min | int | no | 0 | The min value of int data that connector generated |
+| int.max | int | no | 0x7fffffff | The max value of int data that connector generated |
+| int.template | list | no | - | The template list of int type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| bigint.fake.mode | string | no | range | The fake mode of generating bigint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `bigint.template` option |
+| bigint.min | bigint | no | 0 | The min value of bigint data that connector generated |
+| bigint.max | bigint | no | 0x7fffffffffffffff | The max value of bigint data that connector generated |
+| bigint.template | list | no | - | The template list of bigint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| float.fake.mode | string | no | range | The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `float.template` option |
+| float.min | float | no | 0 | The min value of float data that connector generated |
+| float.max | float | no | 0x1.fffffeP+127 | The max value of float data that connector generated |
+| float.template | list | no | - | The template list of float type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| double.fake.mode | string | no | range | The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `double.template` option |
+| double.min | double | no | 0 | The min value of double data that connector generated |
+| double.max | double | no | 0x1.fffffffffffffP+1023 | The max value of double data that connector generated |
+| double.template | list | no | - | The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list |
| table-names | list | no | - |
Review Comment:
description for `table-names`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#issuecomment-1767531006
> Since there have been many changes to the Fake connector recently, it is best for us to merge this PR to the dev branch as soon as possible. Could you update this PR to date? @zhilinli123 cc @ruanwenjun
I made some changes. Thank you for your hard work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#issuecomment-1717189328
PTAL: @Hisoka-X @EricJoy2048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1364824291
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -104,190 +103,91 @@ schema = {
}
```
-### rows
+### Random Generation
-The row list of fake data output per degree of parallelism
-
-example
+> 16 data matching the type are randomly generated
```hocon
-rows = [
- {
- kind = INSERT
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_BEFORE
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_AFTER
- fields = [1, "A_1", 100]
- },
- {
- kind = DELETE
- fields = [1, "A_1", 100]
+source {
+ # This is a example input plugin **only for test and demonstrate the feature input plugin**
+ FakeSource {
+ row.num = 16
+ schema = {
+ fields {
+ c_map = "map<string, string>"
+ c_array = "array<int>"
+ c_string = string
+ c_boolean = boolean
+ c_tinyint = tinyint
+ c_smallint = smallint
+ c_int = int
+ c_bigint = bigint
+ c_float = float
+ c_double = double
+ c_decimal = "decimal(30, 8)"
+ c_null = "null"
+ c_bytes = bytes
+ c_date = date
+ c_timestamp = timestamp
+ }
+ }
+ result_table_name = "fake"
}
-]
+}
```
-### row.num
-
-The total number of data generated per degree of parallelism
-
-### split.num
-
-the number of splits generated by the enumerator for each degree of parallelism
-
-### split.read-interval
-
-The interval(mills) between two split reads in a reader
-
-### map.size
-
-The size of `map` type that connector generated
-
-### array.size
-
-The size of `array` type that connector generated
-
-### bytes.length
-
-The length of `bytes` type that connector generated
-
-### string.length
-
-The length of `string` type that connector generated
-
-### string.fake.mode
-
-The fake mode of generating string data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `string.template` option
-
-### string.template
-
-The template list of string type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### tinyint.fake.mode
-
-The fake mode of generating tinyint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `tinyint.template` option
-
-### tinyint.min
-
-The min value of tinyint data that connector generated
-
-### tinyint.max
-
-The max value of tinyint data that connector generated
-
-### tinyint.template
-
-The template list of tinyint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### smallint.fake.mode
-
-The fake mode of generating smallint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `smallint.template` option
-
-### smallint.min
-
-The min value of smallint data that connector generated
-
-### smallint.max
-
-The max value of smallint data that connector generated
-
-### smallint.template
-
-The template list of smallint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### int.fake.mode
-
-The fake mode of generating int data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `int.template` option
-
-### int.min
-
-The min value of int data that connector generated
-
-### int.max
-
-The max value of int data that connector generated
-
-### int.template
-
-The template list of int type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### bigint.fake.mode
+### Customize the data content Simple:
-The fake mode of generating bigint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `bigint.template` option
+> This is a self-defining data source information, defining whether each piece of data is an add or delete modification operation, and defining what each field stores
-### bigint.min
-
-The min value of bigint data that connector generated
-
-### bigint.max
-
-The max value of bigint data that connector generated
-
-### bigint.template
-
-The template list of bigint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### float.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `float.template` option
-
-### float.min
-
-The min value of float data that connector generated
-
-### float.max
-
-The max value of float data that connector generated
-
-### float.template
-
-The template list of float type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### double.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `double.template` option
-
-### double.min
-
-The min value of double data that connector generated
-
-### double.max
-
-The max value of double data that connector generated
-
-### double.template
-
-The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### table-names
-
-The table list that connector generated, used to simulate multi-table scenarios.
-
-This option will override the `table` option in the `schema` option.
-For example, if you configure the `table-names` option as follows, the connector will generate data for the `test.table1` and `test.table2` tables, the `database.schema.table` will be drop.
Review Comment:
Yes, including https://github.com/apache/seatunnel/pull/5255/files/0e4e841e107b721a73409f10f32ab8b7b7301f0c#diff-368d0e6bf6b91efe5c90ec4eaa163de7332b3e6e64797a0fc6b27dd3d0711326L273-L279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1364833453
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -104,190 +103,91 @@ schema = {
}
```
-### rows
+### Random Generation
-The row list of fake data output per degree of parallelism
-
-example
+> 16 data matching the type are randomly generated
```hocon
-rows = [
- {
- kind = INSERT
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_BEFORE
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_AFTER
- fields = [1, "A_1", 100]
- },
- {
- kind = DELETE
- fields = [1, "A_1", 100]
+source {
+ # This is a example input plugin **only for test and demonstrate the feature input plugin**
+ FakeSource {
+ row.num = 16
+ schema = {
+ fields {
+ c_map = "map<string, string>"
+ c_array = "array<int>"
+ c_string = string
+ c_boolean = boolean
+ c_tinyint = tinyint
+ c_smallint = smallint
+ c_int = int
+ c_bigint = bigint
+ c_float = float
+ c_double = double
+ c_decimal = "decimal(30, 8)"
+ c_null = "null"
+ c_bytes = bytes
+ c_date = date
+ c_timestamp = timestamp
+ }
+ }
+ result_table_name = "fake"
}
-]
+}
```
-### row.num
-
-The total number of data generated per degree of parallelism
-
-### split.num
-
-the number of splits generated by the enumerator for each degree of parallelism
-
-### split.read-interval
-
-The interval(mills) between two split reads in a reader
-
-### map.size
-
-The size of `map` type that connector generated
-
-### array.size
-
-The size of `array` type that connector generated
-
-### bytes.length
-
-The length of `bytes` type that connector generated
-
-### string.length
-
-The length of `string` type that connector generated
-
-### string.fake.mode
-
-The fake mode of generating string data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `string.template` option
-
-### string.template
-
-The template list of string type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### tinyint.fake.mode
-
-The fake mode of generating tinyint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `tinyint.template` option
-
-### tinyint.min
-
-The min value of tinyint data that connector generated
-
-### tinyint.max
-
-The max value of tinyint data that connector generated
-
-### tinyint.template
-
-The template list of tinyint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### smallint.fake.mode
-
-The fake mode of generating smallint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `smallint.template` option
-
-### smallint.min
-
-The min value of smallint data that connector generated
-
-### smallint.max
-
-The max value of smallint data that connector generated
-
-### smallint.template
-
-The template list of smallint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### int.fake.mode
-
-The fake mode of generating int data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `int.template` option
-
-### int.min
-
-The min value of int data that connector generated
-
-### int.max
-
-The max value of int data that connector generated
-
-### int.template
-
-The template list of int type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### bigint.fake.mode
+### Customize the data content Simple:
-The fake mode of generating bigint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `bigint.template` option
+> This is a self-defining data source information, defining whether each piece of data is an add or delete modification operation, and defining what each field stores
-### bigint.min
-
-The min value of bigint data that connector generated
-
-### bigint.max
-
-The max value of bigint data that connector generated
-
-### bigint.template
-
-The template list of bigint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### float.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `float.template` option
-
-### float.min
-
-The min value of float data that connector generated
-
-### float.max
-
-The max value of float data that connector generated
-
-### float.template
-
-The template list of float type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### double.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `double.template` option
-
-### double.min
-
-The min value of double data that connector generated
-
-### double.max
-
-The max value of double data that connector generated
-
-### double.template
-
-The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### table-names
-
-The table list that connector generated, used to simulate multi-table scenarios.
-
-This option will override the `table` option in the `schema` option.
-For example, if you configure the `table-names` option as follows, the connector will generate data for the `test.table1` and `test.table2` tables, the `database.schema.table` will be drop.
Review Comment:
thanks review,complete modification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#issuecomment-1684886637
@hailin0 @ic4y @liugddx run ci thks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] Hisoka-X commented on a diff in pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1306392591
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -103,172 +107,58 @@ schema = {
}
```
-### rows
-
-The row list of fake data output per degree of parallelism
+### Customize the data content Simple:
-example
+> This is a self-defining data source information, defining whether each piece of data is an add or delete modification operation, and defining what each field stores
```hocon
-rows = [
- {
- kind = INSERT
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_BEFORE
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_AFTER
- fields = [1, "A_1", 100]
- },
- {
- kind = DELETE
- fields = [1, "A_1", 100]
+source {
Review Comment:
I remember FakeSource can produce data randomly, could you also add a demo case use that feature? Please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363058878
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
Review Comment:
you missed this.
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
Where is `fields`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363082862
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
This is a sub config of `schema`. Not for `FakeSource` root config. Please refer https://github.com/apache/seatunnel/blob/dev/docs/en/concept/schema-feature.md
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 merged PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 closed pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 closed pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
URL: https://github.com/apache/seatunnel/pull/5255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#issuecomment-1765772084
Since there have been many changes to the Fake connector recently, it is best for us to merge this PR to the dev branch as soon as possible. Could you update this PR to date? @zhilinli123 cc @ruanwenjun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363076337
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
![Uploading cd0ba9dc-0ce2-4c58-86ca-e2785bd38458.jpeg…]()
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
![Uploading cd0ba9dc-0ce2-4c58-86ca-e2785bd38458.jpeg…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363086125
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on a diff in pull request #5255: [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1306818862
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -103,172 +107,58 @@ schema = {
}
```
-### rows
-
-The row list of fake data output per degree of parallelism
+### Customize the data content Simple:
-example
+> This is a self-defining data source information, defining whether each piece of data is an add or delete modification operation, and defining what each field stores
```hocon
-rows = [
- {
- kind = INSERT
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_BEFORE
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_AFTER
- fields = [1, "A_1", 100]
- },
- {
- kind = DELETE
- fields = [1, "A_1", 100]
+source {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1363077409
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| fields | config | yes | - | Define Field information |
Review Comment:
```
### schema [config]
#### fields [Config]
The schema of fake data that you want to generate
#### Examples
```
here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1364790231
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -16,54 +16,53 @@ just for some test cases such as type conversion or connector new feature testin
- [ ] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
-## Options
-
-| name | type | required | default value |
-|---------------------|----------|----------|-------------------------|
-| schema | config | yes | - |
-| rows | config | no | - |
-| row.num | int | no | 5 |
-| split.num | int | no | 1 |
-| split.read-interval | long | no | 1 |
-| map.size | int | no | 5 |
-| array.size | int | no | 5 |
-| bytes.length | int | no | 5 |
-| string.length | int | no | 5 |
-| string.fake.mode | string | no | range |
-| tinyint.fake.mode | string | no | range |
-| tinyint.min | tinyint | no | 0 |
-| tinyint.max | tinyint | no | 127 |
-| tinyint.template | list | no | - |
-| smallint.fake.mode | string | no | range |
-| smallint.min | smallint | no | 0 |
-| smallint.max | smallint | no | 32767 |
-| smallint.template | list | no | - |
-| int.fake.template | string | no | range |
-| int.min | int | no | 0 |
-| int.max | int | no | 0x7fffffff |
-| int.template | list | no | - |
-| bigint.fake.mode | string | no | range |
-| bigint.min | bigint | no | 0 |
-| bigint.max | bigint | no | 0x7fffffffffffffff |
-| bigint.template | list | no | - |
-| float.fake.mode | string | no | range |
-| float.min | float | no | 0 |
-| float.max | float | no | 0x1.fffffeP+127 |
-| float.template | list | no | - |
-| double.fake.mode | string | no | range |
-| double.min | double | no | 0 |
-| double.max | double | no | 0x1.fffffffffffffP+1023 |
-| double.template | list | no | - |
-| table-names | list | no | - |
-| common-options | | no | - |
-
-### schema [config]
-
-#### fields [Config]
-
-The schema of fake data that you want to generate
-
-#### Examples
+## Source Options
+
+| Name | Type | Required | Default | Description |
+|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| schema | config | yes | - | Define Schema information |
+| rows | config | no | - | The row list of fake data output per degree of parallelism |
+| row.num | int | no | 5 | The total number of data generated per degree of parallelism |
+| split.num | int | no | 1 | the number of splits generated by the enumerator for each degree of parallelism |
+| split.read-interval | long | no | 1 | The interval(mills) between two split reads in a reader |
+| map.size | int | no | 5 | The size of `map` type that connector generated |
+| array.size | int | no | 5 | The size of `array` type that connector generated |
+| bytes.length | int | no | 5 | The length of `bytes` type that connector generated |
+| string.length | int | no | 5 | The length of `string` type that connector generated |
+| string.fake.mode | string | no | range | The fake mode of generating string data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `string.template` option |
+| string.template | list | no | - | The template list of string type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| tinyint.fake.mode | string | no | range | The fake mode of generating tinyint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `tinyint.template` option |
+| tinyint.min | tinyint | no | 0 | The min value of tinyint data that connector generated |
+| tinyint.max | tinyint | no | 127 | The max value of tinyint data that connector generated |
+| tinyint.template | list | no | - | The template list of tinyint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| smallint.fake.mode | string | no | range | The fake mode of generating smallint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `smallint.template` option |
+| smallint.min | smallint | no | 0 | The min value of smallint data that connector generated |
+| smallint.max | smallint | no | 32767 | The max value of smallint data that connector generated |
+| smallint.template | list | no | - | The template list of smallint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| int.fake.template | string | no | range | The fake mode of generating int data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `int.template` option |
+| int.min | int | no | 0 | The min value of int data that connector generated |
+| int.max | int | no | 0x7fffffff | The max value of int data that connector generated |
+| int.template | list | no | - | The template list of int type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| bigint.fake.mode | string | no | range | The fake mode of generating bigint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `bigint.template` option |
+| bigint.min | bigint | no | 0 | The min value of bigint data that connector generated |
+| bigint.max | bigint | no | 0x7fffffffffffffff | The max value of bigint data that connector generated |
+| bigint.template | list | no | - | The template list of bigint type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| float.fake.mode | string | no | range | The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `float.template` option |
+| float.min | float | no | 0 | The min value of float data that connector generated |
+| float.max | float | no | 0x1.fffffeP+127 | The max value of float data that connector generated |
+| float.template | list | no | - | The template list of float type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| double.fake.mode | string | no | range | The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `double.template` option |
+| double.min | double | no | 0 | The min value of double data that connector generated |
+| double.max | double | no | 0x1.fffffffffffffP+1023 | The max value of double data that connector generated |
+| double.template | list | no | - | The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list |
+| table-names | list | no | - | The Supports configuration of multiple data sources for example: ["test.table1", "test.table2"] |
Review Comment:
Check the old one https://github.com/apache/seatunnel/pull/5255/files#diff-368d0e6bf6b91efe5c90ec4eaa163de7332b3e6e64797a0fc6b27dd3d0711326L266. You losted many important information in the new one description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation [seatunnel]
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #5255:
URL: https://github.com/apache/seatunnel/pull/5255#discussion_r1364823749
##########
docs/en/connector-v2/source/FakeSource.md:
##########
@@ -104,190 +103,91 @@ schema = {
}
```
-### rows
+### Random Generation
-The row list of fake data output per degree of parallelism
-
-example
+> 16 data matching the type are randomly generated
```hocon
-rows = [
- {
- kind = INSERT
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_BEFORE
- fields = [1, "A", 100]
- },
- {
- kind = UPDATE_AFTER
- fields = [1, "A_1", 100]
- },
- {
- kind = DELETE
- fields = [1, "A_1", 100]
+source {
+ # This is a example input plugin **only for test and demonstrate the feature input plugin**
+ FakeSource {
+ row.num = 16
+ schema = {
+ fields {
+ c_map = "map<string, string>"
+ c_array = "array<int>"
+ c_string = string
+ c_boolean = boolean
+ c_tinyint = tinyint
+ c_smallint = smallint
+ c_int = int
+ c_bigint = bigint
+ c_float = float
+ c_double = double
+ c_decimal = "decimal(30, 8)"
+ c_null = "null"
+ c_bytes = bytes
+ c_date = date
+ c_timestamp = timestamp
+ }
+ }
+ result_table_name = "fake"
}
-]
+}
```
-### row.num
-
-The total number of data generated per degree of parallelism
-
-### split.num
-
-the number of splits generated by the enumerator for each degree of parallelism
-
-### split.read-interval
-
-The interval(mills) between two split reads in a reader
-
-### map.size
-
-The size of `map` type that connector generated
-
-### array.size
-
-The size of `array` type that connector generated
-
-### bytes.length
-
-The length of `bytes` type that connector generated
-
-### string.length
-
-The length of `string` type that connector generated
-
-### string.fake.mode
-
-The fake mode of generating string data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `string.template` option
-
-### string.template
-
-The template list of string type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### tinyint.fake.mode
-
-The fake mode of generating tinyint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `tinyint.template` option
-
-### tinyint.min
-
-The min value of tinyint data that connector generated
-
-### tinyint.max
-
-The max value of tinyint data that connector generated
-
-### tinyint.template
-
-The template list of tinyint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### smallint.fake.mode
-
-The fake mode of generating smallint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `smallint.template` option
-
-### smallint.min
-
-The min value of smallint data that connector generated
-
-### smallint.max
-
-The max value of smallint data that connector generated
-
-### smallint.template
-
-The template list of smallint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### int.fake.mode
-
-The fake mode of generating int data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `int.template` option
-
-### int.min
-
-The min value of int data that connector generated
-
-### int.max
-
-The max value of int data that connector generated
-
-### int.template
-
-The template list of int type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### bigint.fake.mode
+### Customize the data content Simple:
-The fake mode of generating bigint data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `bigint.template` option
+> This is a self-defining data source information, defining whether each piece of data is an add or delete modification operation, and defining what each field stores
-### bigint.min
-
-The min value of bigint data that connector generated
-
-### bigint.max
-
-The max value of bigint data that connector generated
-
-### bigint.template
-
-The template list of bigint type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### float.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `float.template` option
-
-### float.min
-
-The min value of float data that connector generated
-
-### float.max
-
-The max value of float data that connector generated
-
-### float.template
-
-The template list of float type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### double.fake.mode
-
-The fake mode of generating float data, support `range` and `template`, default `range`，if use configured it to `template`, user should also configured `double.template` option
-
-### double.min
-
-The min value of double data that connector generated
-
-### double.max
-
-The max value of double data that connector generated
-
-### double.template
-
-The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list
-
-### table-names
-
-The table list that connector generated, used to simulate multi-table scenarios.
-
-This option will override the `table` option in the `schema` option.
-For example, if you configure the `table-names` option as follows, the connector will generate data for the `test.table1` and `test.table2` tables, the `database.schema.table` will be drop.
Review Comment:
@Hisoka-X here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org