[jira] [Created] (FLINK-30942) Fix the bug that the decided parallelism by adaptive batch scheduler may be larger than the max parallelism

["Lijie Wang (Jira)" <ji...@apache.org>]

Lijie Wang created FLINK-30942:
----------------------------------

             Summary: Fix the bug that the decided parallelism by adaptive batch scheduler may be larger than the max parallelism
                 Key: FLINK-30942
                 URL: https://issues.apache.org/jira/browse/FLINK-30942
             Project: Flink
          Issue Type: Bug
          Components: Runtime / Coordination
            Reporter: Lijie Wang
            Assignee: Lijie Wang
             Fix For: 1.17.0


Currently, when using the adaptive batch scheduler, the vertex parallelism decided by  forward group may be larger than the global max parallelism(which is configured by option {{parallelism.default}} or {{execution.batch.adaptive.auto-parallelism.max-parallelism}}, see FLINK-30686 for details), which will cause the following exception:

{code:java}
Caused by: java.lang.IllegalArgumentException: Vertex's parallelism should be smaller than or equal to vertex's max parallelism.
	at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138)
	at org.apache.flink.runtime.scheduler.DefaultVertexParallelismInfo.setParallelism(DefaultVertexParallelismInfo.java:95)
	at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.setParallelism(ExecutionJobVertex.java:317)
	at org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.changeJobVertexParallelism(AdaptiveBatchScheduler.java:385)
	at org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.initializeVerticesIfPossible(AdaptiveBatchScheduler.java:284)
	at org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.onTaskFinished(AdaptiveBatchScheduler.java:183)
	at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:745)
	at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:725)
	at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:80)
	at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:479)
	... 30 more
{code}





--
This message was sent by Atlassian Jira
(v8.20.10#820010)