You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "loserwang1024 (via GitHub)" <gi...@apache.org> on 2024/08/29 09:01:50 UTC
[PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
loserwang1024 opened a new pull request, #3588:
URL: https://github.com/apache/flink-cdc/pull/3588
CDC yaml without pipeline should not throw exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "loserwang1024 (via GitHub)" <gi...@apache.org>.
loserwang1024 commented on PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#issuecomment-2317097953
@yuxiqian , CC
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "loserwang1024 (via GitHub)" <gi...@apache.org>.
loserwang1024 commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1735899156
##########
flink-cdc-cli/src/test/resources/definitions/pipeline-definition-minimized.yaml:
##########
Review Comment:
Test YamlPipelineDefinitionParserTest.YamlPipelineDefinitionParserTest will do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "loserwang1024 (via GitHub)" <gi...@apache.org>.
loserwang1024 commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1735855282
##########
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java:
##########
@@ -108,8 +108,11 @@ private PipelineDef parse(JsonNode pipelineDefJsonNode, Configuration globalPipe
// UDFs are optional. We parse UDF first and remove it from the pipelineDefJsonNode since
// it's not of plain data types and must be removed before calling toPipelineConfig.
List<UdfDef> udfDefs = new ArrayList<>();
- Optional.ofNullable(((ObjectNode) pipelineDefJsonNode.get(PIPELINE_KEY)).remove(UDF_KEY))
- .ifPresent(node -> node.forEach(udf -> udfDefs.add(toUdfDef(udf))));
+ if (pipelineDefJsonNode.get(PIPELINE_KEY) != null) {
+ Optional.ofNullable(
+ ((ObjectNode) pipelineDefJsonNode.get(PIPELINE_KEY)).remove(UDF_KEY))
+ .ifPresent(node -> node.forEach(udf -> udfDefs.add(toUdfDef(udf))));
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1735850975
##########
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java:
##########
@@ -108,8 +108,11 @@ private PipelineDef parse(JsonNode pipelineDefJsonNode, Configuration globalPipe
// UDFs are optional. We parse UDF first and remove it from the pipelineDefJsonNode since
// it's not of plain data types and must be removed before calling toPipelineConfig.
List<UdfDef> udfDefs = new ArrayList<>();
- Optional.ofNullable(((ObjectNode) pipelineDefJsonNode.get(PIPELINE_KEY)).remove(UDF_KEY))
- .ifPresent(node -> node.forEach(udf -> udfDefs.add(toUdfDef(udf))));
+ if (pipelineDefJsonNode.get(PIPELINE_KEY) != null) {
+ Optional.ofNullable(
+ ((ObjectNode) pipelineDefJsonNode.get(PIPELINE_KEY)).remove(UDF_KEY))
+ .ifPresent(node -> node.forEach(udf -> udfDefs.add(toUdfDef(udf))));
+ }
Review Comment:
a test is welcome,sir
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1735866117
##########
flink-cdc-cli/src/test/resources/definitions/pipeline-definition-minimized.yaml:
##########
Review Comment:
Should `private final PipelineDef minimizedDef` field be updated to reflect this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1739699797
##########
flink-cdc-cli/src/test/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParserTest.java:
##########
@@ -426,7 +427,9 @@ void testParsingFullDefinitionFromString() throws Exception {
Collections.emptyList(),
Collections.emptyList(),
Collections.emptyList(),
- Configuration.fromMap(Collections.singletonMap("parallelism", "1")));
+ Configuration.fromMap(
+ Collections.singletonMap(
+ "local-time-zone", ZoneId.systemDefault().toString())));
Review Comment:
Seems it's not necessary to pass `local-time-zone` explicitly since it will be accessed when constructing `PipelineDef` anyway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on code in PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588#discussion_r1739699797
##########
flink-cdc-cli/src/test/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParserTest.java:
##########
@@ -426,7 +427,9 @@ void testParsingFullDefinitionFromString() throws Exception {
Collections.emptyList(),
Collections.emptyList(),
Collections.emptyList(),
- Configuration.fromMap(Collections.singletonMap("parallelism", "1")));
+ Configuration.fromMap(
+ Collections.singletonMap(
+ "local-time-zone", ZoneId.systemDefault().toString())));
Review Comment:
Seems it's not necessary to pass `local-time-zone` explicitly since it will be added when constructing `PipelineDef` anyway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36174][hotfix] CDC yaml without pipeline should not throw exception. [flink-cdc]
Posted by "ruanhang1993 (via GitHub)" <gi...@apache.org>.
ruanhang1993 merged PR #3588:
URL: https://github.com/apache/flink-cdc/pull/3588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org