You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "xingbe (Jira)" <ji...@apache.org> on 2024/01/18 09:09:00 UTC
[jira] [Created] (FLINK-34143) Modify the effective strategy of `execution.batch.adaptive.auto-parallelism.default-source-parallelism`
xingbe created FLINK-34143:
------------------------------
Summary: Modify the effective strategy of `execution.batch.adaptive.auto-parallelism.default-source-parallelism`
Key: FLINK-34143
URL: https://issues.apache.org/jira/browse/FLINK-34143
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.19.0
Reporter: xingbe
Currently, if users do not set the `{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}} configuration option, the AdaptiveBatchScheduler defaults to a parallelism of 1 for source vertices. In [FLIP-379|https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs#FLIP379:Dynamicsourceparallelisminferenceforbatchjobs-IntroduceDynamicParallelismInferenceinterfaceforSource], the value of `{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}} will act as the upper bound for inferring dynamic source parallelism, and continuing with the current policy is no longer appropriate.
We plan to change the effectiveness strategy of `{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}}; when the user does not set this config option, we will use the value of `{{{}execution.batch.adaptive.auto-parallelism.max-parallelism`{}}} as the upper bound for source parallelism inference. If {{`execution.batch.adaptive.auto-parallelism.max-parallelism`}} is also not configured, the value of `{{{}parallelism.default`{}}} will be used as a fallback.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)