You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by amir bahmanyari <am...@yahoo.com> on 2016/11/11 01:59:15 UTC
TIMED_WAITING KafkaIO() Threads State
I am checking the runtime state of the KafkaIO() threads using jconsole.I see as many cores as I have specified on my server all right.Is TIMED_WAITING the right state for them all to be in all the time?I monitor them and that state never changes...Is the "Total waited" time range reasonable?Thanks+regards,
Re: TIMED_WAITING KafkaIO() Threads State
Posted by Raghu Angadi <ra...@google.com>.
Someone with better knowledge of Flink runner will be able to assist you
better.
Looks like all these threads are waiting for work. I think it will be more
useful if you provide access to stacktraces of all the threads (post a an
external link).
Since there are no threads in 'KafkaUnboundedReader::consumerPollLoop()',
it looks like none of the sources is actually active.
On Fri, Nov 11, 2016 at 5:10 PM, amir bahmanyari <am...@yahoo.com>
wrote:
> Hi Raghu,
> This is text version (of the threads; all look the same) of the image.
> Hope makes some sense as to why it gets so slow...
> Cheers+have a wonderful weekend.
>
> Name: Time Trigger for Source: Read(UnboundedKafkaSource) ->
> AnonymousParDo -> AnonymousParDo (45/64)
> State: TIMED_WAITING on java.util.concurrent.locks.
> AbstractQueuedSynchronizer$ConditionObject@3f752f28
> Total blocked: 8,369 Total waited: 77,763
>
> Stack trace:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$
> ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(
> ScheduledThreadPoolExecutor.java:1093)
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(
> ScheduledThreadPoolExecutor.java:809)
> java.util.concurrent.ThreadPoolExecutor.getTask(
> ThreadPoolExecutor.java:1067)
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1127)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
>
>
>
> ------------------------------
> *From:* Raghu Angadi <ra...@google.com>
> *To:* user@beam.incubator.apache.org; amir bahmanyari <am...@yahoo.com>
>
> *Sent:* Thursday, November 10, 2016 6:20 PM
> *Subject:* Re: TIMED_WAITING KafkaIO() Threads State
>
> Hi Amir,
>
> I suggest you post actual relevant thread stacks here. We can see those in
> the the image, but text is better.
> Essentially Flink UnboundedSource reader is doing a sleep()..
>
> On Thu, Nov 10, 2016 at 5:59 PM, amir bahmanyari <am...@yahoo.com>
> wrote:
>
> I am checking the runtime state of the KafkaIO() threads using jconsole.
> I see as many cores as I have specified on my server all right.
> Is TIMED_WAITING the right state for them all to be in all the time?
> I monitor them and that state never changes...
> Is the "Total waited" time range reasonable?
> Thanks+regards,
>
> [image: Inline image]
>
>
>
>
>
Re: TIMED_WAITING KafkaIO() Threads State
Posted by amir bahmanyari <am...@yahoo.com>.
Hi Raghu,This is text version (of the threads; all look the same) of the image. Hope makes some sense as to why it gets so slow...Cheers+have a wonderful weekend.
Name: Time Trigger for Source: Read(UnboundedKafkaSource) -> AnonymousParDo -> AnonymousParDo (45/64)State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3f752f28Total blocked: 8,369 Total waited: 77,763
Stack trace: sun.misc.Unsafe.park(Native Method)java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)java.lang.Thread.run(Thread.java:745)
From: Raghu Angadi <ra...@google.com>
To: user@beam.incubator.apache.org; amir bahmanyari <am...@yahoo.com>
Sent: Thursday, November 10, 2016 6:20 PM
Subject: Re: TIMED_WAITING KafkaIO() Threads State
Hi Amir,
I suggest you post actual relevant thread stacks here. We can see those in the the image, but text is better. Essentially Flink UnboundedSource reader is doing a sleep()..
On Thu, Nov 10, 2016 at 5:59 PM, amir bahmanyari <am...@yahoo.com> wrote:
I am checking the runtime state of the KafkaIO() threads using jconsole.I see as many cores as I have specified on my server all right.Is TIMED_WAITING the right state for them all to be in all the time?I monitor them and that state never changes...Is the "Total waited" time range reasonable?Thanks+regards,
Re: TIMED_WAITING KafkaIO() Threads State
Posted by Raghu Angadi <ra...@google.com>.
Hi Amir,
I suggest you post actual relevant thread stacks here. We can see those in
the the image, but text is better.
Essentially Flink UnboundedSource reader is doing a sleep()..
On Thu, Nov 10, 2016 at 5:59 PM, amir bahmanyari <am...@yahoo.com>
wrote:
> I am checking the runtime state of the KafkaIO() threads using jconsole.
> I see as many cores as I have specified on my server all right.
> Is TIMED_WAITING the right state for them all to be in all the time?
> I monitor them and that state never changes...
> Is the "Total waited" time range reasonable?
> Thanks+regards,
>
> [image: Inline image]
>
>