Re: [PR] KAFKA-9545: Fix IllegalStateException in updateLags [kafka]

["lucasbru (via GitHub)" <gi...@apache.org>]

lucasbru commented on PR #15096:
URL: https://github.com/apache/kafka/pull/15096#issuecomment-1873828040

   
   > I do not completely understand how the consumer subscription can reflect the topic deletion while Streams has still references to the deleted topic. The update to the subscription of the consumer happens right after the call to `onAssignment()` [1]. Both calls should happen inside the call to `poll()`. What do I miss?
   
   In every iteration of the poll loop of the `ConsumerCoordinator` we update the subscription in `maybeUpdateSubscriptionMetadata`, based on the latest metadata known to the consumer - which will reflect the topic deletion and update the subscription.
   
   The subscription is used to update the assignment in `onJoinPrepare` and `onLeavePrepare`. 
   
   https://github.com/apache/kafka/blob/2b99d0e45027c88ae2347fa8f7d1ff4b2b919089/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L801 
   
   https://github.com/apache/kafka/blob/2b99d0e45027c88ae2347fa8f7d1ff4b2b919089/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L857
   
   The partitions in the `RecordCollector` will only be updated when we have a new assignment, so when in state `PREPARE_SHUTDOWN` or `PARTITIONS_REVOKED` the `RecordCollector` may have `TopicPartition`s that we do not technically own anymore.
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org