You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/10/10 02:55:46 UTC
[GitHub] [incubator-seatunnel] FWLamb opened a new pull request, #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
FWLamb opened a new pull request, #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
Clickhouse Support nest type and array https://github.com/apache/incubator-seatunnel/issues/3014
## Check list
* [x] Code changed are covered with tests, or it does not need tests for reason:
* [x] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1280585148
I want to use **_fieldType_** in the injectFunction to determine what type to use. How can I get it? In other words, there is no public method to obtain the SeatunnelRow metadata At present, I think of a way to add a parameter to the **_injectFields method_** of the **_ClickhouseFieldInjectFunction interface_**. Is there a better way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281767515
> > The clickhouse e2e not open, check this :https://github.com/apache/incubator-seatunnel/pull/3113/files. Add it in your PR, so we can make sure the e2e without problems.
>
> Add both or just the clickhouse?
just clickhouse
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1280143698
PTAL. The task https://github.com/apache/incubator-seatunnel/issues/3101 need to use the current pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r996683087
##########
seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/common/serialization/InternalRowConverter.java:
##########
@@ -216,4 +213,31 @@ private static SeaTunnelRow reconvert(InternalRow engineRow, SeaTunnelRowType ro
}
return new SeaTunnelRow(fields);
}
+
+ private static Object reconvertArray(Object field, SeaTunnelDataType<?> dataType) {
+ BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
+ Object[] objectArray = ((ArrayData) field).toObjectArray(TypeConverterUtils.convert(elementType));
+ switch (elementType.getSqlType()) {
Review Comment:
But it will affects all sinks, can you handle it in clickhouse sink alone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997694080
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
I mean you should remove `spark-unsafe_2.11`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281777076
PR updated, PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1280944018
>  I want to use **_fieldType_** in the injectFunction to determine what type to use. How can I get it? In other words, there is no public method to obtain the SeatunnelRow metadata At present, I think of a way to add a parameter to the **_injectFields method_** of the **_ClickhouseFieldInjectFunction interface_**. Is there a better way?
You can just add a parameter in `ArrayInjectFunction` contructer. But if you do that, you should make sure two different array field can't use same `ArrayInjectFunction` instance like now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997692621
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
> Connector are without engine, can't add it in dependency.
Okay, I removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r996653946
##########
seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/common/serialization/InternalRowConverter.java:
##########
@@ -216,4 +213,31 @@ private static SeaTunnelRow reconvert(InternalRow engineRow, SeaTunnelRowType ro
}
return new SeaTunnelRow(fields);
}
+
+ private static Object reconvertArray(Object field, SeaTunnelDataType<?> dataType) {
+ BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
+ Object[] objectArray = ((ArrayData) field).toObjectArray(TypeConverterUtils.convert(elementType));
+ switch (elementType.getSqlType()) {
Review Comment:
> Why we need copy data from array to another array?
Because we need to convert the object type array to the exact wrapper type array according to the SQL type. If we do not do this, when JDBC inserts the array data, it will return ClassCastException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997696437
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
> Can you tell me your wechat number?
541947595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281765687
> The clickhouse e2e not open, check this :https://github.com/apache/incubator-seatunnel/pull/3113/files. Add it in your PR, so we can make sure the e2e without problems.
Add both or just the clickhouse?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r996718615
##########
seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/common/serialization/InternalRowConverter.java:
##########
@@ -216,4 +213,31 @@ private static SeaTunnelRow reconvert(InternalRow engineRow, SeaTunnelRowType ro
}
return new SeaTunnelRow(fields);
}
+
+ private static Object reconvertArray(Object field, SeaTunnelDataType<?> dataType) {
+ BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
+ Object[] objectArray = ((ArrayData) field).toObjectArray(TypeConverterUtils.convert(elementType));
+ switch (elementType.getSqlType()) {
Review Comment:
I have some worry about performance. Any way, the logic is fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997693530
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
I know you want deal with array(String), because
```
ArrayData arrayData = (ArrayData) field;
BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
return arrayData.toObjectArray(TypeConverterUtils.convert(elementType));
```
can't work right.
Maybe you can use recursion to handle this special case. In InternalRowConverter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997693530
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
I know you want deal with array(String), because
```
ArrayData arrayData = (ArrayData) field;
BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
return arrayData.toObjectArray(TypeConverterUtils.convert(elementType));
```
can't work right.
Maybe you can use recursion to handle this special case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r996712646
##########
seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/common/serialization/InternalRowConverter.java:
##########
@@ -216,4 +213,31 @@ private static SeaTunnelRow reconvert(InternalRow engineRow, SeaTunnelRowType ro
}
return new SeaTunnelRow(fields);
}
+
+ private static Object reconvertArray(Object field, SeaTunnelDataType<?> dataType) {
+ BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
+ Object[] objectArray = ((ArrayData) field).toObjectArray(TypeConverterUtils.convert(elementType));
+ switch (elementType.getSqlType()) {
Review Comment:
I think other connectors can also work. Compared with the previous ones, they are actually the downward transformation of object arrays. The core logic has not changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1280586057
@Hisoka-X
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1280946332
Or add set type method into `ArrayInjectFunction`, not change constructer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1290466066
Retry after #3183 Merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r996579247
##########
seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/common/serialization/InternalRowConverter.java:
##########
@@ -216,4 +213,31 @@ private static SeaTunnelRow reconvert(InternalRow engineRow, SeaTunnelRowType ro
}
return new SeaTunnelRow(fields);
}
+
+ private static Object reconvertArray(Object field, SeaTunnelDataType<?> dataType) {
+ BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
+ Object[] objectArray = ((ArrayData) field).toObjectArray(TypeConverterUtils.convert(elementType));
+ switch (elementType.getSqlType()) {
Review Comment:
Why we need copy data from array to another array?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281738407
> >  I want to use **_fieldType_** in the injectFunction to determine what type to use. How can I get it? In other words, there is no public method to obtain the SeatunnelRow metadata At present, I think of a way to add a parameter to the **_injectFields method_** of the **_ClickhouseFieldInjectFunction interface_**. Is there a better way?
>
> Are you will update this logic in this PR?
Yes, I add a member variable in ArrayInjectFunction and initialize it in isCurrentFieldType when appropriate. I will update this pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997693530
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
I know you want deal with array<String>, because
```
ArrayData arrayData = (ArrayData) field;
BasicType<?> elementType = ((ArrayType<?, ?>) dataType).getElementType();
return arrayData.toObjectArray(TypeConverterUtils.convert(elementType));
```
can't work right.
Maybe you can use recursion to handle this special case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1272817811
> The Clickhouse Source also not support array type, please check `org.apache.seatunnel.connectors.seatunnel.clickhouse.util.TypeConvertUtil:75`.
Forget it, I find it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997691105
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
Connector are without engine, can't add it in dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281735935
>  I want to use **_fieldType_** in the injectFunction to determine what type to use. How can I get it? In other words, there is no public method to obtain the SeatunnelRow metadata At present, I think of a way to add a parameter to the **_injectFields method_** of the **_ClickhouseFieldInjectFunction interface_**. Is there a better way?
Are you will update this logic in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1281757426
The clickhouse e2e not open, check this :https://github.com/apache/incubator-seatunnel/pull/3113/files. Add it in your PR, so we can make sure the e2e without problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997695633
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
Can you tell me your wechat number?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
FWLamb commented on code in PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#discussion_r997696437
##########
seatunnel-connectors-v2/connector-clickhouse/pom.xml:
##########
@@ -61,6 +61,12 @@
<artifactId>clickhouse-jdbc</artifactId>
<version>${clickhouse.version}</version>
</dependency>
+ <dependency>
Review Comment:
> Can you tell me your wechat number?
541947595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047#issuecomment-1272815342
The Clickhouse Source also not support array type, please check `org.apache.seatunnel.connectors.seatunnel.clickhouse.util.TypeConvertUtil:75`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer merged pull request #3047: [Improve] [Connector-V2-Clickhouse] Support nest type and array
Posted by GitBox <gi...@apache.org>.
TyrantLucifer merged PR #3047:
URL: https://github.com/apache/incubator-seatunnel/pull/3047
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org