You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Andrey Zagrebin (JIRA)" <ji...@apache.org> on 2019/07/22 15:44:00 UTC

[jira] [Updated] (FLINK-13371) Release partitions in JM if producer restarts

     [ https://issues.apache.org/jira/browse/FLINK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrey Zagrebin updated FLINK-13371:
------------------------------------
    Summary: Release partitions in JM if producer restarts  (was: Release partitions in JM if producer gets restarted)

> Release partitions in JM if producer restarts
> ---------------------------------------------
>
>                 Key: FLINK-13371
>                 URL: https://issues.apache.org/jira/browse/FLINK-13371
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination, Runtime / Network
>    Affects Versions: 1.9.0
>            Reporter: Andrey Zagrebin
>            Priority: Major
>
> As discussed in FLINK-13245, there can be a case that producer does not even detect any consumption attempt if consumer fails before the connection is established. It means we cannot fully rely on shuffle service for the release on consumption in case of consumer failure. When producer restarts it will leak partitions from the previous attempt. Previously we had an explicit release call for this case in Execution.cancel/suspend. Basically JM has to explicitly release all partitions produced by the previous task execution attempt in case of producer restart, including `released on consumption` partitions. For this change, we might need to track all partitions in PartitionTrackerImpl.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)