You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "xingbe (Jira)" <ji...@apache.org> on 2023/12/07 02:02:00 UTC

[jira] [Updated] (FLINK-33768) Support dynamic source parallelism inference for batch jobs

     [ https://issues.apache.org/jira/browse/FLINK-33768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

xingbe updated FLINK-33768:
---------------------------
    Description: 
Currently, for JobVertices without parallelism configured, the AdaptiveBatchScheduler dynamically infers the vertex parallelism based on the volume of input data. Specifically, for Source vertices, it uses the value of `{*}execution.batch.adaptive.auto-parallelism.default-source-parallelism{*}` as the fixed parallelism. If this is not set by the user, the default value of {{1}}  is used as the source parallelism, which is actually a temporary implementation solution.

We aim to support dynamic source parallelism inference for batch jobs. More details see [FLIP-379|https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs].

  was:Support dynamic source parallelism inference for batch jobs, 


> Support dynamic source parallelism inference for batch jobs
> -----------------------------------------------------------
>
>                 Key: FLINK-33768
>                 URL: https://issues.apache.org/jira/browse/FLINK-33768
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Coordination
>    Affects Versions: 1.19.0
>            Reporter: xingbe
>            Priority: Major
>
> Currently, for JobVertices without parallelism configured, the AdaptiveBatchScheduler dynamically infers the vertex parallelism based on the volume of input data. Specifically, for Source vertices, it uses the value of `{*}execution.batch.adaptive.auto-parallelism.default-source-parallelism{*}` as the fixed parallelism. If this is not set by the user, the default value of {{1}}  is used as the source parallelism, which is actually a temporary implementation solution.
> We aim to support dynamic source parallelism inference for batch jobs. More details see [FLIP-379|https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)