You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ChaomingZhangCN (via GitHub)" <gi...@apache.org> on 2025/01/23 09:55:46 UTC

[PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

ChaomingZhangCN opened a new pull request, #3887:
URL: https://github.com/apache/flink-cdc/pull/3887

   https://issues.apache.org/jira/browse/FLINK-35713


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "ChaomingZhangCN (via GitHub)" <gi...@apache.org>.
ChaomingZhangCN commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818044280

   ping @leonardBang


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "ChaomingZhangCN (via GitHub)" <gi...@apache.org>.
ChaomingZhangCN commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2910966285

   @leonardBang PTAL~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2829629612

   CI failed...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818105857

   Minor suggestion: shall we pick `MySqlToPaimonE2eITCase` and `FlinkParallelizedPipelineITCase` to be executed with different source / sink parallelism? I believe they could cover most parallelism-aware operators (flush event alignment, 2-PC sinks, and schema operators).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on code in PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#discussion_r2052201271


##########
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java:
##########
@@ -188,7 +189,14 @@ private SourceDef toSourceDef(JsonNode sourceNode) {
         // "name" field is optional
         String name = sourceMap.remove(NAME_KEY);
 
-        return new SourceDef(type, name, Configuration.fromMap(sourceMap));
+        // "parallelism" field is optional
+        Integer parallelismValue = null;

Review Comment:
   Could we use `pipeline.parallelism`'s valued as  default parallelism instead of an error-prone null value? then we  can use `int` type and do not need to use `@Nullable ` annotation in related methods.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "ChaomingZhangCN (via GitHub)" <gi...@apache.org>.
ChaomingZhangCN closed pull request #3887: [FLINK-35713][common] Support set parallelism in source and sink
URL: https://github.com/apache/flink-cdc/pull/3887


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818053249

   @ChaomingZhangCN Could you rebase this PR to latest master branch as it bumped junit version to junit5 for all tests?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2942419571

   @ChaomingZhangCN Could you fix the conflicts?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "ChaomingZhangCN (via GitHub)" <gi...@apache.org>.
ChaomingZhangCN commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2611679804

   @lvyanquan @leonardBang PTAL.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

Posted by "ChaomingZhangCN (via GitHub)" <gi...@apache.org>.
ChaomingZhangCN commented on PR #3887:
URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2742331251

   @yuxiqian I found that the PR-[3438](https://github.com/apache/flink-cdc/pull/3438/files) has been closed, could you please take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org