You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Seshadri, Balaji" <Ba...@dish.com> on 2013/08/14 22:50:04 UTC
Blocking on Consumer Iterator blocking queue
We are getting hanged on consumer side when we try to consume data using Scala API.
We found it when we did load test.Attaching thread dump,please let us know if we have any fix.
Thanks,
Balaji
RE: Blocking on Consumer Iterator blocking queue
Posted by "Seshadri, Balaji" <Ba...@dish.com>.
We also suspect this is because of KAFKA-914 bug which was not part of our client build.
So we are rebuilding with latest Kafka Source.
-----Original Message-----
From: Seshadri, Balaji
Sent: Wednesday, August 14, 2013 10:53 PM
To: users@kafka.apache.org
Subject: RE: Blocking on Consumer Iterator blocking queue
We see WAITING state in all our thread dumps.
We did see data in Kafka but consumer is not able to consume.
-----Original Message-----
From: Seshadri, Balaji [mailto:Balaji.Seshadri@dish.com]
Sent: Wednesday, August 14, 2013 8:29 PM
To: users@kafka.apache.org
Subject: RE: Blocking on Consumer Iterator blocking queue
pool-3-thread-26" Id=74 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@424fc8ad
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:63)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:61)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:53)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
-----Original Message-----
From: Joel Koshy [mailto:jjkoshy.w@gmail.com]
Sent: Wednesday, August 14, 2013 6:01 PM
To: users@kafka.apache.org
Subject: Re: Blocking on Consumer Iterator blocking queue
I don't think this mailing list takes attachments. Can you provide more details and paste your threaddump?
Thanks,
Joel
On Wed, Aug 14, 2013 at 1:50 PM, Seshadri, Balaji <Ba...@dish.com> wrote:
> We are getting hanged on consumer side when we try to consume data
> using Scala API.
>
>
> We found it when we did load test.Attaching thread dump,please let us
> know if we have any fix.
>
>
>
> Thanks,
>
>
>
> Balaji
>
>
RE: Blocking on Consumer Iterator blocking queue
Posted by "Seshadri, Balaji" <Ba...@dish.com>.
We see WAITING state in all our thread dumps.
We did see data in Kafka but consumer is not able to consume.
-----Original Message-----
From: Seshadri, Balaji [mailto:Balaji.Seshadri@dish.com]
Sent: Wednesday, August 14, 2013 8:29 PM
To: users@kafka.apache.org
Subject: RE: Blocking on Consumer Iterator blocking queue
pool-3-thread-26" Id=74 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@424fc8ad
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:63)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:61)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:53)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
-----Original Message-----
From: Joel Koshy [mailto:jjkoshy.w@gmail.com]
Sent: Wednesday, August 14, 2013 6:01 PM
To: users@kafka.apache.org
Subject: Re: Blocking on Consumer Iterator blocking queue
I don't think this mailing list takes attachments. Can you provide more details and paste your threaddump?
Thanks,
Joel
On Wed, Aug 14, 2013 at 1:50 PM, Seshadri, Balaji <Ba...@dish.com> wrote:
> We are getting hanged on consumer side when we try to consume data
> using Scala API.
>
>
> We found it when we did load test.Attaching thread dump,please let us
> know if we have any fix.
>
>
>
> Thanks,
>
>
>
> Balaji
>
>
RE: Blocking on Consumer Iterator blocking queue
Posted by "Seshadri, Balaji" <Ba...@dish.com>.
pool-3-thread-26" Id=74 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@424fc8ad
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:63)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:61)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:53)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
-----Original Message-----
From: Joel Koshy [mailto:jjkoshy.w@gmail.com]
Sent: Wednesday, August 14, 2013 6:01 PM
To: users@kafka.apache.org
Subject: Re: Blocking on Consumer Iterator blocking queue
I don't think this mailing list takes attachments. Can you provide more details and paste your threaddump?
Thanks,
Joel
On Wed, Aug 14, 2013 at 1:50 PM, Seshadri, Balaji <Ba...@dish.com> wrote:
> We are getting hanged on consumer side when we try to consume data
> using Scala API.
>
>
> We found it when we did load test.Attaching thread dump,please let us
> know if we have any fix.
>
>
>
> Thanks,
>
>
>
> Balaji
>
>
Re: Blocking on Consumer Iterator blocking queue
Posted by Joel Koshy <jj...@gmail.com>.
I don't think this mailing list takes attachments. Can you provide
more details and paste your threaddump?
Thanks,
Joel
On Wed, Aug 14, 2013 at 1:50 PM, Seshadri, Balaji
<Ba...@dish.com> wrote:
> We are getting hanged on consumer side when we try to consume data using
> Scala API.
>
>
> We found it when we did load test.Attaching thread dump,please let us know
> if we have any fix.
>
>
>
> Thanks,
>
>
>
> Balaji
>
>
Re: Blocking on Consumer Iterator blocking queue
Posted by Jun Rao <ju...@gmail.com>.
Have you looked at
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F?
Thanks,
Jun
On Wed, Aug 14, 2013 at 1:50 PM, Seshadri, Balaji
<Ba...@dish.com>wrote:
> We are getting hanged on consumer side when we try to consume data using
> Scala API.****
>
>
> We found it when we did load test.Attaching thread dump,please let us
> know if we have any fix.****
>
> ** **
>
> Thanks,****
>
> ** **
>
> Balaji****
>
> ** **
>