You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Fabian Paul (Jira)" <ji...@apache.org> on 2021/09/28 12:52:00 UTC

[jira] [Commented] (FLINK-24398) KafkaSourceFetcherManager should re-use an existing SplitFetcher to commit offset if possible

    [ https://issues.apache.org/jira/browse/FLINK-24398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421366#comment-17421366 ] 

Fabian Paul commented on FLINK-24398:
-------------------------------------

After looking through the code a bit more I am not sure that calling getRunningFetcher() instead of accessing the fetchers directly with fetchers.get(0) changes much here. AFAICT there is always exactly one fetcher and both calls would return null and create a new fetcher. In general, I think it is safer to call getRunningFetcher() because it does not rely on the implicit information that 0 is the only fetcher id.

 

[~renqs] wdyt?

> KafkaSourceFetcherManager should re-use an existing SplitFetcher to commit offset if possible
> ---------------------------------------------------------------------------------------------
>
>                 Key: FLINK-24398
>                 URL: https://issues.apache.org/jira/browse/FLINK-24398
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Dong Lin
>            Priority: Minor
>              Labels: pull-request-available
>
> Currently KafkaSourceFetcherManager::commitOffset() will create a new SplitFetcher if the fetchers.get(0) == null. As a result, if the first fetcher has already been closed and removed, N fetchers will be created for N commitOffset() invocations.
> A more efficient approach is to re-use an existing fetcher to commit the offset if there is any running fetchers in this KafkaSourceFetcherManager.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)