[jira] [Created] (KAFKA-19781) Consumer NoOffsetForPartitionException for partitions being revoked

["Lianet Magrans (Jira)" <ji...@apache.org>]

Lianet Magrans created KAFKA-19781:
--------------------------------------

             Summary: Consumer NoOffsetForPartitionException for partitions being revoked
                 Key: KAFKA-19781
                 URL: https://issues.apache.org/jira/browse/KAFKA-19781
             Project: Kafka
          Issue Type: Bug
          Components: clients, consumer
            Reporter: Lianet Magrans
            Assignee: Lianet Magrans
             Fix For: 4.2.0


Currently, the consumer attempts to update positions for all partitions it owns that don't have a position, even when the partition may be already being revoked. This could lead to NoOffsetForPartitionException for such partitions (if there is no reset strategy defined). There are 2 main issues around this:
 * is probably unneeded work to fetch offsets and update positions for partitions being revoked, and at that point we're actually not allowing fetching from there anymore
 * throwing NoOffsetForPartitionException may lead applications to take action to set positions themselves, which will most probably fail (the partition will most probably not owned by the consumer anymore since it was already being revoked when the NoOffsetForPartitionException was generated). We've seen this on Kafka Streams, that handled NoOffsetForPartitionException by calling seek to set positions.   

This task is to review if there are no other implications I may be missing? Then fix to ensure we don't update positions for partitions being revoked (aligned with how we don't allow fetching from them)

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)