You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2022/08/15 07:07:07 UTC
[GitHub] [inlong] Jellal-HT opened a new pull request, #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Jellal-HT opened a new pull request, #5547:
URL: https://github.com/apache/inlong/pull/5547
### Prepare a Pull Request
- Fixes #5099
### Motivation
Support Apache Hudi in sort module and manager module
### Modifications
- Extend a new Extract Node for Apache Hudi
- Extend a new Load Node for Apache Hudi
- Add the corresponding flink connectors for Apache Hudi
- Extend Extract Node and Load Node in manager module for apache Hudi
### Verifying this change
- [x] This change added tests and can be verified as follows:
- add the Unit test of HudiLoadNode HudiLoadNodeTest
- add the Unit test of HudiExtractNode HudiExtractNodeTest
- add Hudi source service test HudiSourceServiceTest for manager module
- add Hudi sink service test HudiSinkServiceTest for manager module
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang commented on pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on PR #5547:
URL: https://github.com/apache/inlong/pull/5547#issuecomment-1343794519
duplicated with https://github.com/apache/inlong/pull/6789 and https://github.com/apache/inlong/pull/6790, close it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang commented on pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on PR #5547:
URL: https://github.com/apache/inlong/pull/5547#issuecomment-1214824089
@Jellal-HT great job. please add the license file for the new dependency, and you can refer to https://inlong.apache.org/zh-CN/community/how-to-maintain-3rd-party-dependencies/.
otherwise, the workflow runs fail. Please check again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang closed pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
dockerzhang closed pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
URL: https://github.com/apache/inlong/pull/5547
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] XuQianJin-Stars commented on pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #5547:
URL: https://github.com/apache/inlong/pull/5547#issuecomment-1221972942
hi @Jellal-HT A small suggested version upgrade to 0.12 for hudi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] gong commented on a diff in pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
gong commented on code in PR #5547:
URL: https://github.com/apache/inlong/pull/5547#discussion_r951125586
##########
inlong-sort/sort-common/src/main/java/org/apache/inlong/sort/protocol/node/load/HudiLoadNode.java:
##########
@@ -72,6 +79,7 @@ public Map<String, String> tableOptions() {
Map<String, String> options = super.tableOptions();
options.put("connector", "inlong-hudi");
Review Comment:
Hi, connector is 'inlong-hudi', but I can't find code in sort-connector-hudi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] duowan1520 commented on pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
duowan1520 commented on PR #5547:
URL: https://github.com/apache/inlong/pull/5547#issuecomment-1225620770
Are you missing the step of bundling extract node and load node into the InLong sort ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] ZuoFengZhang commented on a diff in pull request #5547: [INLONG-5460][Sort][Manager] Support Apache Hudi
Posted by GitBox <gi...@apache.org>.
ZuoFengZhang commented on code in PR #5547:
URL: https://github.com/apache/inlong/pull/5547#discussion_r1043296573
##########
inlong-sort/sort-common/src/main/java/org/apache/inlong/sort/protocol/node/load/HudiLoadNode.java:
##########
@@ -72,6 +79,7 @@ public Map<String, String> tableOptions() {
Map<String, String> options = super.tableOptions();
options.put("connector", "inlong-hudi");
Review Comment:
Thks, I have already dealt with it in another PR, will close this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org