[jira] [Closed] (FLINK-20721) Remove unknown input channels and process to update partitions

["Zhu Zhu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-20721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhu Zhu closed FLINK-20721.
---------------------------
    Resolution: Won't Do

Closing this ticket because UnknownInputChannel are still needed at the moment if BLOCKING edges exist in a pipelined region. 

e.g.
A --pipelined-->B, A --pipelined-->C, B --pipelined-->D, C --blocking-->D.

In this case, even though D knows the location of C when deployed, the corresponding input channel will be unknown because C has not finished producing the BLOCKING result partition. This is needed because reading from an uncompleted BLOCKING result partition is not supported yet.

Besides that, UnknownInputChannel is possibly needed for future individual failover.

> Remove unknown input channels and process to update partitions
> --------------------------------------------------------------
>
>                 Key: FLINK-20721
>                 URL: https://issues.apache.org/jira/browse/FLINK-20721
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Coordination, Runtime / Network
>    Affects Versions: 1.13.0
>            Reporter: Zhu Zhu
>            Assignee: Zhu Zhu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.13.0
>
>
> With the latest pipelined region scheduling, Flink no longer launch a task before knowing the locations of all the partitions it consumes. `scheduleOrUpdateConsumers` is no longer needed and is removed in FLINK-20439.
> Unknown input channels and the process to update it is also no longer needed. I'd propose to remove them and the benefits are:
> 1. simplifying the code of both scheduler and shuffle components
> 2. simplifying interfaces of ShuffleEnvironment and ShuffleDescriptor 
> 3. ensure the assumptions in InputGate#resumeConsumption() implementations
> 4. allow to remove ScheduleMode#allowLazyDeployment() and later completely remove ScheduleMode



--
This message was sent by Atlassian Jira
(v8.3.4#803005)