[jira] [Commented] (KAFKA-1958) ZookeeperConsumerConnector doesn't remove consumer node on shutdown.

["Beletsky Andrey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14322765#comment-14322765 ] 

Beletsky Andrey commented on KAFKA-1958:
----------------------------------------

+Reason of it:+
{code}
ZookeeperConsumerConnector.shutdown() ->
      ConsumerFetcherManager.stopConnections() ->
                    leaderFinderThread.shutdown() - fails with InterruptedException
{code}

Stack trace:
{noformat}
java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301) ~[na:1.7.0_65]
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236) ~[na:1.7.0_65]
        at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) ~[kafka_2.10-0.8.1.1.jar:na]
        at kafka.consumer.ConsumerFetcherManager.stopConnections(ConsumerFetcherManager.scala:143) ~[kafka_2.10-0.8.1.1.jar:na]
        at kafka.consumer.ZookeeperConsumerConnector.liftedTree1$1(ZookeeperConsumerConnector.scala:171) [kafka_2.10-0.8.1.1.jar:na]
        at kafka.consumer.ZookeeperConsumerConnector.shutdown(ZookeeperConsumerConnector.scala:167) [kafka_2.10-0.8.1.1.jar:na]
        at kafka.javaapi.consumer.ZookeeperConsumerConnector.shutdown(ZookeeperConsumerConnector.scala:110) [kafka_2.10-0.8.1.1.jar:na]
{noformat}


I think ZookeeperConsumerConnector should be smart and correctly close own resources (threads), release partition ownership, remove consumer Id from /consumer/myGroup/ids/<consumer_id>.

> ZookeeperConsumerConnector doesn't remove consumer node on shutdown.
> --------------------------------------------------------------------
>
>                 Key: KAFKA-1958
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1958
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 0.8.1.1
>            Reporter: Beletsky Andrey
>            Assignee: Neha Narkhede
>              Labels: consumer, shutdown, zookeeper
>
> We use kafka with ZooKeeper via high level consumer.
> There is a scheduled job that creates a consumer with specific group, does necessary logic and shuts down this consumer.
> +An issue:+
> Nobody deletes */consumers/myGroup/ids/myGroup_<ip>_<postfix>*. And after several job runs there are a lot of dead consumer IDs under myGroup. I've got [an issue|https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage?] that new consumer doesn't see a partition.
> We start to implement an approach to remove a consumer nodes from Zookeeper manually after consumer is shutted down.
> I think better way to remove this node during *ZookeeperConsumerConnector.shutdown()*.
> *P.S.:*
> If I missed something in your sources please let me know.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)