You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Dong Lin (Jira)" <ji...@apache.org> on 2021/09/28 12:40:00 UTC

[jira] [Created] (FLINK-24398) KafkaSourceFetcherManager should re-use an existing SplitFetcher to commit offset if possible

Dong Lin created FLINK-24398:
--------------------------------

             Summary: KafkaSourceFetcherManager should re-use an existing SplitFetcher to commit offset if possible
                 Key: FLINK-24398
                 URL: https://issues.apache.org/jira/browse/FLINK-24398
             Project: Flink
          Issue Type: Improvement
            Reporter: Dong Lin


Currently KafkaSourceFetcherManager::commitOffset() will create a new SplitFetcher if the fetchers.get(0) == null. As a result, if the first fetcher has already been closed and removed, N fetchers will be created for N commitOffset() invocations.

A more efficient approach is to re-use an existing fetcher to commit the offset if there is any running fetchers in this KafkaSourceFetcherManager.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)