You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Andrey Zagrebin (JIRA)" <ji...@apache.org> on 2019/07/31 07:12:00 UTC
[jira] [Commented] (FLINK-13493) BoundedBlockingSubpartition only
notifies onConsumedSubpartition when all the readers are empty
[ https://issues.apache.org/jira/browse/FLINK-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896834#comment-16896834 ]
Andrey Zagrebin commented on FLINK-13493:
-----------------------------------------
As I think, we also discussed this with [~StephanEwen]. Now it should not be a problem because the existing connection should always end one way or another, as we already discussed in FLINK-13478, and the existing readers should be all closed, eventually triggering the file deletion. We also switched from counters to boolean flags per subpartition to avoid counting of unconsumed subpartitions by multiple reader confirmation from other subpartitions. Eventually, we do not need the release on consumption for blocking partitions, it stays as a safety net at the moment to switch to the previous behaviour. At the moment this issue looks like for me the same as concerns described FLINK-13478.
> BoundedBlockingSubpartition only notifies onConsumedSubpartition when all the readers are empty
> -----------------------------------------------------------------------------------------------
>
> Key: FLINK-13493
> URL: https://issues.apache.org/jira/browse/FLINK-13493
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Minor
>
> In previous implementation, it would always notify the {{ResultPartition}} of consumed subpartition if any reader view is released. Based on the reference-counter release strategy it might cause problems if one blocking subpartition has multiple readers. That means the whole result partition might be released but there are still alive readers in some subpartitions.
> Although the default release strategy for blocking partition is based on JM/scheduler notification atm. But if we switch the option to based on consumption notification it would cause problems. And from the subpartition side it should has the right behavior no matter what is the specific release strategy in upper layer.
> In order to fix this bug, the {{BoundedBlockingSubpartition}} would only notify {{onConsumedSubpartition}} when all the readers are empty.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)