You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Mason Chen (Jira)" <ji...@apache.org> on 2022/07/08 06:16:00 UTC
[jira] [Commented] (FLINK-27479) HybridSource refreshes availability future
[ https://issues.apache.org/jira/browse/FLINK-27479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564100#comment-17564100 ]
Mason Chen commented on FLINK-27479:
------------------------------------
I was able to reproduce the user's reported issue in https://lists.apache.org/thread/l6m9p4cnmm4dzcm7nr285g2xyhnw96c1. Please see the PR description for more details. Sorry for the delay!
> HybridSource refreshes availability future
> ------------------------------------------
>
> Key: FLINK-27479
> URL: https://issues.apache.org/jira/browse/FLINK-27479
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Affects Versions: 1.14.4
> Reporter: Mason Chen
> Assignee: Mason Chen
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.2
>
> Attachments: hybrid-source-with-fix.html, hybrid-source-without-fix.html, kafka-source.html
>
>
> HybridSourceReader needs to refresh the availability future according to the underlying reader. It currently maintains its own future and completes it after the sub-reader's availability future is complete. However, the implementation does not refresh the future again until the reader receives a switch event. This can cause a tight loop with the Flink runtime repeatedly invoking pollNext() and high CPU utilization.
>
> To solve this, we can reuse the MultipleFuturesAvailabilityHelper to manage the lifecycle of the availability future.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)