[jira] [Commented] (KAFKA-17448) New consumer seek should update positions in background thread

["Kirk True (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17879653#comment-17879653 ] 

Kirk True commented on KAFKA-17448:
-----------------------------------

[~yangpoan]—sorry for the nag, but can you mark this as Patch Available since the PR is out for review? Thanks!

> New consumer seek should update positions in background thread
> --------------------------------------------------------------
>
>                 Key: KAFKA-17448
>                 URL: https://issues.apache.org/jira/browse/KAFKA-17448
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, consumer
>    Affects Versions: 3.7.0, 3.8.0, 3.7.1
>            Reporter: Lianet Magrans
>            Assignee: PoAn Yang
>            Priority: Major
>              Labels: consumer-threading-refactor, kip-848-client-support
>             Fix For: 4.0.0
>
>
> In the new AsyncKafkaConsumer, a call to seek will update the positions in subscription state for the assigned partitions in the app thread ([https://github.com/apache/kafka/blob/c23b6b0365af5c58b76d8ad3fb628f766f95348f/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L796])
> This could lead to race conditions like we've seen when subscription state changes in the app thread (over a set of assigned partitions), that could have been modified in the background thread, leading to errors on "No current assignment for partition " [https://github.com/apache/kafka/blob/c23b6b0365af5c58b76d8ad3fb628f766f95348f/clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java#L378] 
> Also, positions update is moved the background with KAFKA-17066 for the same reason, so even if the assignment does not change, we could have a race between the background setting positions to the committed offsets for instance, and the app thread setting them manually via seek. 
> To avoid all of the above, we should have seek generate an event, send it to the background, and then update the subscription state when processing that event (similar to other api calls, ex, assign with KAFKA-17064)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)