[jira] [Created] (KAFKA-20106) Improve behaviour of consumer assignment API under async reconciliations

["Lianet Magrans (Jira)" <ji...@apache.org>]

Lianet Magrans created KAFKA-20106:
--------------------------------------

             Summary: Improve behaviour of consumer assignment API under async reconciliations
                 Key: KAFKA-20106
                 URL: https://issues.apache.org/jira/browse/KAFKA-20106
             Project: Kafka
          Issue Type: Task
          Components: clients, consumer
            Reporter: Lianet Magrans
            Assignee: Lianet Magrans


With the new rebalance protocol and the AsyncConsumer implementation, the client reconciles assignments in the background as it received them from the broker.

This leads to new interactions of the assignment API with others like seek() or position(). Taking seek as an example, given that the assignment can change in the background in between calls to assignment() and seek(), applications can hit IllegalStateException "No current assignment for partition t0" on seek() if the partition got unassigned. This is part of the contract of the seek() call, but note that under the classic protocol and consumer, it requires a call to poll to make anything change in the assignment (so seek could only fail in this way if there is a call to poll in-between, but not on sequence of assignment() + seek())

One option is to change how to update the subscription state when a reconciliation completes, so that even if the reconciliation completes in the background it's not made effective until there is call to poll. Main consideration here would be if there is any impact, e.g  atm partitions become fetchable as soon as the reconciliation completes, so wonder if this would change/affect the partitions that get added to a fetch request in the background (even though we only generate fetch requests on the foreground on poll)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)