You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Raja Pragada <pr...@gmail.com> on 2016/11/15 04:01:57 UTC

KafkaConsumer hangs after partition assignment

I am running a 0.9.0.1 Java Kafka Consumer client with 10 consumer threads.
When ever Partition is revoked/assigned, all the consumers hang in the same
state and don't receive any more records.
It typically runs for 30 mins. or so after which it hangs. I am running
with "auto.commit" set to *false*.

Here is the full code :
https://gist.github.com/vissapra/363da2acb6c150e3102b19542a042c9b

All threads hang with this state: Only a restart fixes it and they will
 java.lang.Thread.State: RUNNABLE
 at sun.nio.ch.KQueueArrayWrapper.kevent0(KQueueArrayWrapper.java:-1)re is f
 at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
 at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
 at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
 - locked <0x131d> (a sun.nio.ch.KQueueSelectorImpl)
 - locked <0x131e> (a java.util.Collections$UnmodifiableSet)
 - locked <0x131f> (a sun.nio.ch.Util$2)
 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
 at org.apache.kafka.common.network.Selector.select(Selector.java:425)
 at org.apache.kafka.common.network.Selector.poll(Selector.java:254)
 at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256)
 at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
 at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
 at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
 at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:134)
 at
org.apache.kafka.clients.consumer.internals.Fetcher.listOffset(Fetcher.java:317)
 at
org.apache.kafka.clients.consumer.internals.Fetcher.resetOffset(Fetcher.java:291)
 at
org.apache.kafka.clients.consumer.internals.Fetcher.updateFetchPositions(Fetcher.java:167)
 at
org.apache.kafka.clients.consumer.KafkaConsumer.updateFetchPositions(KafkaConsumer.java:1302)
 at
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:895)
 at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:853)

Can you suggest where am I getting wrong? I am using 0.9 version of Kafka
server and client.

Thanks
Raja