[jira] [Created] (KAFKA-6397) Consumer should not block setting initial positions of unavailable partitions

["Jason Gustafson (JIRA)" <ji...@apache.org>]

Jason Gustafson created KAFKA-6397:
--------------------------------------

             Summary: Consumer should not block setting initial positions of unavailable partitions
                 Key: KAFKA-6397
                 URL: https://issues.apache.org/jira/browse/KAFKA-6397
             Project: Kafka
          Issue Type: Bug
            Reporter: Jason Gustafson
            Assignee: Jason Gustafson


Currently the consumer will block in poll() after receiving its assignment in order to set the starting offset for every assigned partition. If the topic is deleted or if a partition is unavailable, the consumer can be stuck indefinitely. Most of the time this is not a problem since the starting offset is obtained from the committed offsets, which does not depend on partition availability. However, if there are no committed offsets or if the user has manually called {{seekToBeginning}} or {{seekToEnd}}, then we will need to do a lookup for the starting offset from the partition leader, which will stall the consumer until the partition is available or recreated. It would be better to let the consumer fetch on partitions which are available and periodically check availability for the rest. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)