You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by xt...@apache.org on 2022/12/22 09:35:54 UTC
[flink] branch master updated (4ea67f63eb1 -> a56a4d38d1f)
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
from 4ea67f63eb1 [FLINK-30188][coordination] Set partition finished state in ConsumedPartitionGroup for dynamic graph correctly.
add f83575a19a2 [hotfix] Fix the incorrect assumption in HsResultPartition that result partition always invoke close before release.
add 90473d8b213 [hotfix] TestingSchedulingResultPartition is no longer mark partition finished when register consumed partition group.
add bf1cb5fe789 [FLINK-30233] Introduce AllFinishedInputConsumableDecider.
add ca8f5cf93c6 [FLINK-30233] execution vertex supports mark hybrid result partition finished.
add a56a4d38d1f [FLINK-30233] Speculative scheduler supports hybrid result partition.
No new revisions were added by this update.
Summary of changes:
.../generated/all_jobmanager_section.html | 6 ++
.../generated/expert_scheduling_section.html | 6 ++
.../generated/job_manager_configuration.html | 6 ++
.../e5126cae-f3fe-48aa-b6fb-60ae6cc3fcd5 | 4 +-
.../flink/configuration/JobManagerOptions.java | 20 ++++++
.../executiongraph/DefaultExecutionGraph.java | 12 +++-
.../DefaultExecutionGraphBuilder.java | 6 +-
.../flink/runtime/executiongraph/Execution.java | 2 +-
.../runtime/executiongraph/ExecutionVertex.java | 26 +++++---
.../IntermediateResultPartition.java | 6 +-
.../InternalExecutionGraphAccessor.java | 2 +
.../MarkPartitionFinishedStrategy.java} | 26 +++-----
.../partition/hybrid/HsMemoryDataManager.java | 17 +++--
.../partition/hybrid/HsMemoryDataSpiller.java | 16 +----
.../partition/hybrid/HsResultPartition.java | 5 +-
.../scheduler/DefaultExecutionGraphFactory.java | 5 +-
.../runtime/scheduler/ExecutionGraphFactory.java | 2 +
.../flink/runtime/scheduler/SchedulerBase.java | 8 +++
.../scheduler/adaptive/AdaptiveScheduler.java | 4 ++
.../adaptivebatch/AdaptiveBatchScheduler.java | 19 +++++-
.../AdaptiveBatchSchedulerFactory.java | 67 +++++++++++++++----
.../adaptivebatch/SpeculativeScheduler.java | 6 +-
.../AllFinishedInputConsumableDecider.java | 66 +++++++++++++++++++
.../scheduler/strategy/ConsumedPartitionGroup.java | 2 -
.../strategy/DefaultInputConsumableDecider.java | 2 +-
.../strategy/VertexwiseSchedulingStrategy.java | 10 ++-
.../TestingDefaultExecutionGraphBuilder.java | 4 +-
.../partition/hybrid/HsMemoryDataSpillerTest.java | 18 +-----
.../DefaultExecutionGraphFactoryTest.java | 2 +
.../runtime/scheduler/DefaultSchedulerBuilder.java | 14 ++--
.../adapter/DefaultExecutionTopologyTest.java | 2 +-
.../runtime/scheduler/adaptive/ExecutingTest.java | 7 ++
.../adaptivebatch/SpeculativeSchedulerTest.java | 19 ++++--
...hedulingDownstreamTasksInBatchJobBenchmark.java | 4 +-
... => AllFinishedInputConsumableDeciderTest.java} | 75 +++++-----------------
.../PipelinedRegionSchedulingStrategyTest.java | 8 +--
.../strategy/TestingSchedulingResultPartition.java | 4 --
37 files changed, 332 insertions(+), 176 deletions(-)
copy flink-runtime/src/main/java/org/apache/flink/runtime/{topology/Result.java => executiongraph/MarkPartitionFinishedStrategy.java} (64%)
create mode 100644 flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/strategy/AllFinishedInputConsumableDecider.java
copy flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/strategy/{DefaultInputConsumableDeciderTest.java => AllFinishedInputConsumableDeciderTest.java} (63%)