[jira] [Commented] (FLINK-27479) HybridSource refreshes availability future

["Selim Abidin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-27479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575145#comment-17575145 ] 

Selim Abidin commented on FLINK-27479:
--------------------------------------

This problem exists in 1.13.6 versions too. 

> HybridSource refreshes availability future
> ------------------------------------------
>
>                 Key: FLINK-27479
>                 URL: https://issues.apache.org/jira/browse/FLINK-27479
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Common
>    Affects Versions: 1.14.4
>            Reporter: Mason Chen
>            Assignee: Mason Chen
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.16.0, 1.15.2
>
>         Attachments: hybrid-source-with-fix.html, hybrid-source-without-fix.html, kafka-source.html
>
>
> HybridSourceReader needs to refresh the availability future according to the underlying reader. It currently maintains its own future and completes it after the sub-reader's availability future is complete. However, the implementation does not refresh the future again until the reader receives a switch event. This can cause a tight loop with the Flink runtime repeatedly invoking pollNext() and high CPU utilization.
>  
> To solve this, we can reuse the MultipleFuturesAvailabilityHelper to manage the lifecycle of the availability future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)