[jira] [Updated] (KAFKA-6397) Consumer should not block setting initial positions of unavailable partitions

["Damian Guy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Damian Guy updated KAFKA-6397:
------------------------------
    Fix Version/s:     (was: 1.1.0)
                   1.2.0

> Consumer should not block setting initial positions of unavailable partitions
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-6397
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6397
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Gustafson
>            Assignee: Jason Gustafson
>            Priority: Major
>              Labels: consumer
>             Fix For: 1.2.0
>
>
> Currently the consumer will block in poll() after receiving its assignment in order to set the starting offset for every assigned partition. If the topic is deleted or if a partition is unavailable, the consumer can be stuck indefinitely. Most of the time this is not a problem since the starting offset is obtained from the committed offsets, which does not depend on partition availability. However, if there are no committed offsets or if the user has manually called {{seekToBeginning}} or {{seekToEnd}}, then we will need to do a lookup for the starting offset from the partition leader, which will stall the consumer until the partition is available or recreated. It would be better to let the consumer fetch on partitions which are available and periodically check availability for the rest. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)