You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/13 03:50:22 UTC
[GitHub] [incubator-seatunnel] 1141086028 opened a new pull request, #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
1141086028 opened a new pull request, #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170
1. Add FlinkSQL support for MySQL CDC
you can see https://github.com/ververica/flink-cdc-connectors
2. I tested the SET configuration and found that the "'" configuration did not take effect. So I removed the "'" of set in the "flink.sql.conf.template" test example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1185250820
Hi, @legendtkl，I have finished adding.
I will make as much contribution as possible ~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1202982214
> hi, @1141086028 , this PR need to rebase the dev branch to resolve the failed check.
hi, @legendtkl , I have finished rebase operation, please check again. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1186144504
Hi, @1141086028 , the conflicts should be resolved first, and the jars in [known-dependencies.txt](https://github.com/apache/incubator-seatunnel/blob/dev/tools/dependencies/known-dependencies.txt) should be arranged with dictionary order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1263378247
Hi, what's news about this pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1277053709
I will close this pr because it has not been updated for too long. If necessary, please reopen it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1196182024
hi, @1141086028 , this PR need to rebase the dev branch to resolve the failed check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1186845757
Hi, @1141086028 , please check the failed check step "Backend / Dependency licenses" , it seems a missing dependency jar is needed https://github.com/apache/incubator-seatunnel/runs/7382295880?check_suite_focus=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1190219293
> > Hi, @1141086028 , please check the failed check step "Backend / Dependency licenses" , it seems a missing dependency jar is needed https://github.com/apache/incubator-seatunnel/runs/7382295880?check_suite_focus=true
>
> Hi, @legendtkl, I checked the error log information and found that "org.apache.seatunnel:connector-common:jar:2.1.3-SNAPSHOT" was missing, but I didn't reference this jar. I didn't know how to solve this problem
Hi, @1141086028 , I checked the error log. The failure is caused by the jar "zstd-jni-1.5.2-1.jar" you added. I think you can remove this and try again.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1192454204
> > > Hi, @1141086028 , please check the failed check step "Backend / Dependency licenses" , it seems a missing dependency jar is needed https://github.com/apache/incubator-seatunnel/runs/7382295880?check_suite_focus=true
> >
> >
> > Hi, @legendtkl, I checked the error log information and found that "org.apache.seatunnel:connector-common:jar:2.1.3-SNAPSHOT" was missing, but I didn't reference this jar. I didn't know how to solve this problem
>
> Hi, @1141086028 , I checked the error log. The failure is caused by the jar "zstd-jni-1.5.2-1.jar" you added. I think you can remove this and try again.
>
> Thanks.
Hi,@legendtkl, I have removed the jar "zstd-jni-1.5.2-1.jar" . Please check it again.
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1186706774
Hi, @legendtkl. I have sorted by dictionary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ashulin commented on a diff in pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
ashulin commented on code in PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#discussion_r924012705
##########
seatunnel-core/seatunnel-core-flink-sql/pom.xml:
##########
@@ -60,6 +60,16 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.ververica</groupId>
+ <artifactId>flink-sql-connector-mysql-cdc</artifactId>
+ <version>${flink.cdc.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-runtime-web_${scala.binary.version}</artifactId>
+ <version>${flink.version}</version>
+ </dependency>
Review Comment:
```suggestion
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime-web_${scala.binary.version}</artifactId>
</dependency>
```
nit: The dependencies in `<dependencyManagement>` are referenced here, `<version>` can be omitted
##########
seatunnel-core/seatunnel-core-flink-sql/pom.xml:
##########
@@ -60,6 +60,16 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.ververica</groupId>
+ <artifactId>flink-sql-connector-mysql-cdc</artifactId>
+ <version>${flink.cdc.version}</version>
+ </dependency>
Review Comment:
`flink-sql-connector-*` dependency should be added to `seatunnel-connectors-flink-sql-dist` module
##########
seatunnel-examples/seatunnel-flink-sql-examples/pom.xml:
##########
@@ -95,5 +95,11 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</dependency>
+
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-runtime-web_${scala.binary.version}</artifactId>
+ <version>${flink.version}</version>
+ </dependency>
Review Comment:
```suggestion
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime-web_${scala.binary.version}</artifactId>
</dependency>
```
Same the above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1189877617
> Hi, @1141086028 , please check the failed check step "Backend / Dependency licenses" , it seems a missing dependency jar is needed https://github.com/apache/incubator-seatunnel/runs/7382295880?check_suite_focus=true
Hi, @legendtkl, I checked the error log information and found that "org.apache.seatunnel:connector-common:jar:2.1.3-SNAPSHOT" was missing, but I didn't reference this jar. I didn't know how to solve this problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1183977132
Hi, @1141086028 , LGTM with a minor comment.
Dependency check failed with detailed message https://github.com/apache/incubator-seatunnel/runs/7314242930?check_suite_focus=true
It could be fixed by adding jar `flink-sql-connector-mysql-cdc-2.1.1.jar` into file https://github.com/apache/incubator-seatunnel/blob/dev/tools/dependencies/known-dependencies.txt
Look forward to your more contribute about Flink SQL module ~
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1183924377
Hi @legendtkl PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 closed pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
EricJoy2048 closed pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
URL: https://github.com/apache/incubator-seatunnel/pull/2170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
legendtkl commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1192511445
@Hisoka-X would you help to approve running workflow? It seems the workflow is blocked by Github mechanism.
<img width="993" alt="image" src="https://user-images.githubusercontent.com/2370761/180436866-ad35d1a8-bf16-417e-852f-bb3dfa04c8ff.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1193043811
Some CI problem will be fixed in #2242. After that PR merged, then CI can check dependency license normally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 1141086028 commented on pull request #2170: [Feature][Flink SQL CDC] Add "MySQL CDC Source Connector"
Posted by GitBox <gi...@apache.org>.
1141086028 commented on PR #2170:
URL: https://github.com/apache/incubator-seatunnel/pull/2170#issuecomment-1189877096
Hi, @legendtkl, I checked the error log information and found that "org.apache.seatunnel:connector-common:jar:2.1.3-SNAPSHOT" was missing, but I didn't reference this jar. I didn't know how to solve this problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org