You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Alex (JIRA)" <ji...@apache.org> on 2019/07/24 09:09:00 UTC
[jira] [Comment Edited] (FLINK-13384) The back pressure monitoring
does not work for StreamSources
[ https://issues.apache.org/jira/browse/FLINK-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891700#comment-16891700 ]
Alex edited comment on FLINK-13384 at 7/24/19 9:08 AM:
-------------------------------------------------------
> FLINK-12483 introduced a separate thread for executing the sources in the mailbox model.
I confirm, for the streaming source tasks, the source function (and other chained operators) is now running in a separate thread.
For such tasks, the main task thread now mostly waiting for termination of that thread and does task shutdown/cleanup.
I'm not aware of any explicit changes/adjustments related to backpressure tracking.
Quick look into {{BackPressureStatsTrackerImpl}} source code, points to tracking {{LocalBufferPool.requestBufferBlocking}} method calls in stack traces. It's possible that this method is not used anymore in the production (due to some other changes to make inputs non-blocking).
was (Author: 1u0):
> FLINK-12483 introduced a separate thread for executing the sources in the mailbox model.
I confirm, for the streaming source tasks, the source function (and other chained operators) is now running in a separate thread.
For such tasks, the main task thread now mostly waiting for termination of that thread and does task shutdown/cleanup.
I'm not aware of any explicit changes/adjustments related to backpressure tracking.
Quick look into {{BackPressureStatsTrackerImpl}} source code, points to tracking {{LocalBufferPool.requestBufferBlocking}} method calls in stack traces. It's possible that this method is not used anymore in the production (due to some other changes making inputs less blocking).
> The back pressure monitoring does not work for StreamSources
> ------------------------------------------------------------
>
> Key: FLINK-13384
> URL: https://issues.apache.org/jira/browse/FLINK-13384
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics, Runtime / Task
> Affects Versions: 1.9.0
> Reporter: Dawid Wysakowicz
> Assignee: Dawid Wysakowicz
> Priority: Blocker
> Fix For: 1.9.0
>
>
> I think it is caused by: [FLINK-12483]. The reason is that the {{BackPressureStatsTrackerImpl}} samples only the main thread. FLINK-12483 introduced a separate thread for executing the sources in the mailbox model. It is similar to other old bug that concerned only Kafka source: [FLINK-3456]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)