[jira] [Updated] (KAFKA-17505) New consumer seekToBeginning/End should run in background thread

["Lianet Magrans (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-17505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lianet Magrans updated KAFKA-17505:
-----------------------------------
    Labels: consumer-threading-refactor kip-848-client-support  (was: kip-848-client-support)

> New consumer seekToBeginning/End should run in background thread
> ----------------------------------------------------------------
>
>                 Key: KAFKA-17505
>                 URL: https://issues.apache.org/jira/browse/KAFKA-17505
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, consumer
>            Reporter: Lianet Magrans
>            Assignee: TaiJuWu
>            Priority: Major
>              Labels: consumer-threading-refactor, kip-848-client-support
>             Fix For: 4.0.0
>
>
> seekToBeginning and seekToEnd perform 2 operations:
>  # retrieve assigned partitions from the subscription state if no partitions provided by the user
>  # request offsets reset for those partitions (to EARLIEST or LATEST offset) (update partition state in the subscription state)
> In the new asyn consumer, these operation are performed sequentially in the app thread, which could lead to errors if retrieving the assigned partitions first, and then requesting the reset: the assignment could change in the background thread in-between 1) and 2) and the reset would fail with
> "No current assignment for partition"
>  
> We should move the implementation to the background thread, where all modifications to the subscription state should occur. We could introduce a new event (something like OffsetResetEvent or similar) that would take as input the set of partitions received in the api calls and the OffsetResetStrategy (EARLIEST or LATEST). When processing it in the background, it would safely perform the same 2 actions that are currently in the app thread seekToBeginning/End.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)