[jira] [Updated] (KAFKA-832) 0.8 Consumer prevents rebalance if consumer thread is blocked or slow

["Jun Rao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao updated KAFKA-832:
--------------------------

    Attachment: kafka-832.patch

Attach a patch, which addresses the deadlock in the following way.

1. AbstractFetcherThread: make the lock acquiring in add/remove partition interruptible.

2. ConsumerFetcherManager: To stop connections, we first stop the leaderFinder thread and then stop all fetchers. The stopping of the leaderFinder thread uses interrupt and is expected to be non-blocking because of the changes make in item 1.
                
> 0.8 Consumer prevents rebalance if consumer thread is blocked or slow
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-832
>                 URL: https://issues.apache.org/jira/browse/KAFKA-832
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Jun Rao
>            Priority: Blocker
>              Labels: kafka-0.8, p2
>         Attachments: kafka-832.patch
>
>
> 0.8 consumer involves multiple locks across the fetcher manager, leader finder thread and the rebalance watcher executor that prevent rebalance from interrupting the consumer. Rebalance should always take precedence over consumption, not the other way around. The effect of this bug is that if a consumer is lagging and the fetcher thread is blocked on adding the next fetched data chunk to the internal blocking queue, rebalance is halted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira