You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhu Zhu (JIRA)" <ji...@apache.org> on 2019/04/10 06:10:00 UTC
[jira] [Commented] (FLINK-11776) Refactor to simplify the process
of scheduleOrUpdateConsumers
[ https://issues.apache.org/jira/browse/FLINK-11776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814081#comment-16814081 ]
Zhu Zhu commented on FLINK-11776:
---------------------------------
Nice catch. With this change duplicated partitions updates can be avoided as well.
I think it can also be part of https://issues.apache.org/jira/browse/FLINK-12036.
> Refactor to simplify the process of scheduleOrUpdateConsumers
> -------------------------------------------------------------
>
> Key: FLINK-11776
> URL: https://issues.apache.org/jira/browse/FLINK-11776
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Based on the work of [FLINK-11417|https://issues.apache.org/jira/browse/FLINK-11417], the process of {{Execution#scheduleOrUpdateConsumers}} do not need to consider the race condition with schedule process, so we can refactor to make it easy to handle.
> * The concurrent data structure for cacheing partial input channel descriptor can be replaced by common list.
> * If the consumer is in {{CREATED}} state, we only need to schedule it and the partition info would be known during deployment.
> * If the consumer is in {{SCHEDULED}} state, we need do nothing.
> * If the consumer is in {{RUNNING}} state, we can send partition info immediately.
> * If the consumer is in {{DEPLOYING}} state, we can cache the partition info in order to send them in batch after consumer switching to {{RUNNING}} state.
> * {{PartialInputChannelDeploymentDescriptor}} is not needed any more, we can cache partition info directly.
> This refactoring is also a preparation work for future introducing {{ShuffleMaster}} in [FLINK-11391|https://issues.apache.org/jira/browse/FLINK-11391].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)