You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Sagar Naik <sn...@attributor.com> on 2008/11/20 06:48:38 UTC
Hadoop 18.1 "ls" stalls
*Problem:*
The "ls" is taking noticeable time to respond.
*system:*
I have about 1.6 million files and namenode is prety much full with
heap(2400MB).
I have configured dfs.handler.count to 100, and all IPC server threads
are locked onto a Queue (I think callQueue)
In the namenode logs, I occasionally see Out of memory Exception.
Any suggestions, to improve the response time of system ?
I can increase the dfs.handler.count to 256, but I guess I will see more
OOM exceptions.
Re: Hadoop 18.1 "ls" stalls
Posted by Sagar Naik <sn...@attributor.com>.
Unfortunately, I am not getting it now, bcoz we have turned off our
services. and ( I cant start them immediately)
But I used to get retryInvocationHandler (or similar) in stack
and ls tht took 4-5 secs had listed 37 files only.
It was not lsr option.
Thats what surprised me.
-Sagar
Raghu Angadi wrote:
> Sagar Naik wrote:
>> Thanks Raghu,
>> *datapoints:*
>> - So when I use FSShell client, it gets into retry mode for
>> getFilesInfo() call and takes a long time.
>
> What does "retry mode" mean?
>
>> - Also, when do a "ls" operation, it takes secs(4/5) .
>>
>> - 1.6 million files and namenode is mostly full with heap(2400M)
>> (from ui)
>
> When you say 'ls', how many does it return? (ie. ls of one file, or
> -lsr of thousands of files etc).
>
> None of the IPC threads in your stack trace is doing any work.
>
>
Re: Hadoop 18.1 "ls" stalls
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
Sagar Naik wrote:
> Thanks Raghu,
> *datapoints:*
> - So when I use FSShell client, it gets into retry mode for
> getFilesInfo() call and takes a long time.
What does "retry mode" mean?
> - Also, when do a "ls" operation, it takes secs(4/5) .
>
> - 1.6 million files and namenode is mostly full with heap(2400M) (from
> ui)
When you say 'ls', how many does it return? (ie. ls of one file, or -lsr
of thousands of files etc).
None of the IPC threads in your stack trace is doing any work.
Re: Hadoop 18.1 "ls" stalls
Posted by Sagar Naik <sn...@attributor.com>.
Thanks Raghu,
*datapoints:*
- So when I use FSShell client, it gets into retry mode for
getFilesInfo() call and takes a long time.
- Also, when do a "ls" operation, it takes secs(4/5) .
- 1.6 million files and namenode is mostly full with heap(2400M)
(from ui)
The jstack is
2008-11-20 11:24:27
Full thread dump Java HotSpot(TM) Server VM (10.0-b19 mixed mode):
"Attach Listener" daemon prio=10 tid=0x0a7e6000 nid=0x538a waiting on
condition [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"SocketListener0-2" prio=10 tid=0x13336800 nid=0x6bc8 in Object.wait()
[0x14f67000..0x14f67ed0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x48ae6df0> (a org.mortbay.util.ThreadPool$PoolThread)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:522)
- locked <0x48ae6df0> (a org.mortbay.util.ThreadPool$PoolThread)
Locked ownable synchronizers:
- None
"Trash Emptier" daemon prio=10 tid=0x17070800 nid=0x428 waiting on
condition [0x14f17000..0x14f171d0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.fs.Trash$Emptier.run(Trash.java:219)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.DFSClient$LeaseChecker@159d510" daemon prio=10
tid=0x17070400 nid=0x427 sleeping[0x14fb9000..0x14fb9150]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.dfs.DFSClient$LeaseChecker.run(DFSClient.java:792)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"IPC Server handler 99 on 54310" daemon prio=10 tid=0x17061400 nid=0x41f
waiting on condition [0x15009000..0x1500a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 98 on 54310" daemon prio=10 tid=0x09956000 nid=0x41e
waiting on condition [0x1505b000..0x1505b1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 97 on 54310" daemon prio=10 tid=0x08b36400 nid=0x41d
waiting on condition [0x150ab000..0x150ac150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 96 on 54310" daemon prio=10 tid=0x17418400 nid=0x41c
waiting on condition [0x150fc000..0x150fced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 95 on 54310" daemon prio=10 tid=0x1705dc00 nid=0x41b
waiting on condition [0x1514d000..0x1514de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 94 on 54310" daemon prio=10 tid=0x1705cc00 nid=0x41a
waiting on condition [0x1519e000..0x1519efd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 93 on 54310" daemon prio=10 tid=0x1705a800 nid=0x419
waiting on condition [0x151ef000..0x151eff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 92 on 54310" daemon prio=10 tid=0x17059800 nid=0x418
waiting on condition [0x15240000..0x152410d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 91 on 54310" daemon prio=10 tid=0x17058400 nid=0x417
waiting on condition [0x15291000..0x15292050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 90 on 54310" daemon prio=10 tid=0x17057400 nid=0x416
waiting on condition [0x152e3000..0x152e31d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 89 on 54310" daemon prio=10 tid=0x17056000 nid=0x415
waiting on condition [0x15333000..0x15334150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 88 on 54310" daemon prio=10 tid=0x17055000 nid=0x414
waiting on condition [0x15384000..0x15384ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 87 on 54310" daemon prio=10 tid=0x17053c00 nid=0x413
waiting on condition [0x153d5000..0x153d5e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 86 on 54310" daemon prio=10 tid=0x17052800 nid=0x412
waiting on condition [0x15426000..0x15426fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 85 on 54310" daemon prio=10 tid=0x17051800 nid=0x411
waiting on condition [0x15477000..0x15477f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 84 on 54310" daemon prio=10 tid=0x17050400 nid=0x410
waiting on condition [0x154c8000..0x154c90d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 83 on 54310" daemon prio=10 tid=0x1704f400 nid=0x40f
waiting on condition [0x15519000..0x1551a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 82 on 54310" daemon prio=10 tid=0x1704e400 nid=0x40e
waiting on condition [0x1556b000..0x1556b1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 81 on 54310" daemon prio=10 tid=0x1704d000 nid=0x40d
waiting on condition [0x155bb000..0x155bc150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 80 on 54310" daemon prio=10 tid=0x1704c000 nid=0x40c
waiting on condition [0x1560c000..0x1560ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 79 on 54310" daemon prio=10 tid=0x1704ac00 nid=0x40b
waiting on condition [0x1565d000..0x1565de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 78 on 54310" daemon prio=10 tid=0x17049c00 nid=0x40a
waiting on condition [0x156ae000..0x156aefd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 77 on 54310" daemon prio=10 tid=0x17048800 nid=0x409
waiting on condition [0x156ff000..0x156fff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 76 on 54310" daemon prio=10 tid=0x17047800 nid=0x408
waiting on condition [0x15750000..0x157510d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 75 on 54310" daemon prio=10 tid=0x17046400 nid=0x407
waiting on condition [0x157a1000..0x157a2050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 74 on 54310" daemon prio=10 tid=0x17045400 nid=0x406
waiting on condition [0x157f3000..0x157f31d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 73 on 54310" daemon prio=10 tid=0x17044000 nid=0x405
waiting on condition [0x15843000..0x15844150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 72 on 54310" daemon prio=10 tid=0x17043000 nid=0x404
waiting on condition [0x15894000..0x15894ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 71 on 54310" daemon prio=10 tid=0x17041c00 nid=0x403
waiting on condition [0x158e5000..0x158e5e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 70 on 54310" daemon prio=10 tid=0x17040c00 nid=0x402
waiting on condition [0x15936000..0x15936fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 69 on 54310" daemon prio=10 tid=0x1703f800 nid=0x401
waiting on condition [0x15987000..0x15987f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 68 on 54310" daemon prio=10 tid=0x1703e800 nid=0x400
waiting on condition [0x159d8000..0x159d90d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 67 on 54310" daemon prio=10 tid=0x1703d400 nid=0x3ff
waiting on condition [0x15a29000..0x15a2a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 66 on 54310" daemon prio=10 tid=0x1703c400 nid=0x3fe
waiting on condition [0x15a7b000..0x15a7b1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 65 on 54310" daemon prio=10 tid=0x1703b000 nid=0x3fd
waiting on condition [0x15acb000..0x15acc150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 64 on 54310" daemon prio=10 tid=0x1703a000 nid=0x3fc
waiting on condition [0x15b1c000..0x15b1ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 63 on 54310" daemon prio=10 tid=0x17038c00 nid=0x3fb
waiting on condition [0x15b6d000..0x15b6de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 62 on 54310" daemon prio=10 tid=0x17037c00 nid=0x3fa
waiting on condition [0x15bbe000..0x15bbefd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 61 on 54310" daemon prio=10 tid=0x17036800 nid=0x3f9
waiting on condition [0x15c0f000..0x15c0ff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 60 on 54310" daemon prio=10 tid=0x17035800 nid=0x3f8
waiting on condition [0x15c60000..0x15c610d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 59 on 54310" daemon prio=10 tid=0x17034400 nid=0x3f7
waiting on condition [0x15cb1000..0x15cb2050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 58 on 54310" daemon prio=10 tid=0x17033400 nid=0x3f6
waiting on condition [0x15d03000..0x15d031d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 57 on 54310" daemon prio=10 tid=0x17032000 nid=0x3f5
waiting on condition [0x15d53000..0x15d54150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 56 on 54310" daemon prio=10 tid=0x17031000 nid=0x3f4
waiting on condition [0x15da4000..0x15da4ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 55 on 54310" daemon prio=10 tid=0x1702fc00 nid=0x3f3
waiting on condition [0x15df5000..0x15df5e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 54 on 54310" daemon prio=10 tid=0x1702ec00 nid=0x3f2
waiting on condition [0x15e46000..0x15e46fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 53 on 54310" daemon prio=10 tid=0x1702d800 nid=0x3f1
waiting on condition [0x15e97000..0x15e97f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 52 on 54310" daemon prio=10 tid=0x1702c800 nid=0x3f0
waiting on condition [0x15ee8000..0x15ee90d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 51 on 54310" daemon prio=10 tid=0x1702b800 nid=0x3ef
waiting on condition [0x15f39000..0x15f3a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 50 on 54310" daemon prio=10 tid=0x1702a400 nid=0x3ee
waiting on condition [0x15f8b000..0x15f8b1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 49 on 54310" daemon prio=10 tid=0x17029400 nid=0x3ed
waiting on condition [0x15fdb000..0x15fdc150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 48 on 54310" daemon prio=10 tid=0x17028000 nid=0x3ec
waiting on condition [0x1602c000..0x1602ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 47 on 54310" daemon prio=10 tid=0x17027000 nid=0x3eb
waiting on condition [0x1607d000..0x1607de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 46 on 54310" daemon prio=10 tid=0x173d1400 nid=0x3ea
waiting on condition [0x160ce000..0x160cefd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 45 on 54310" daemon prio=10 tid=0x173d0000 nid=0x3e9
waiting on condition [0x1611f000..0x1611ff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 44 on 54310" daemon prio=10 tid=0x173cf000 nid=0x3e8
waiting on condition [0x16170000..0x161710d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 43 on 54310" daemon prio=10 tid=0x173cdc00 nid=0x3e7
waiting on condition [0x161c1000..0x161c2050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 42 on 54310" daemon prio=10 tid=0x173ccc00 nid=0x3e6
waiting on condition [0x16213000..0x162131d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 41 on 54310" daemon prio=10 tid=0x173cb800 nid=0x3e5
waiting on condition [0x16263000..0x16264150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 40 on 54310" daemon prio=10 tid=0x173ca800 nid=0x3e4
waiting on condition [0x162b4000..0x162b4ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 39 on 54310" daemon prio=10 tid=0x17373800 nid=0x3e3
waiting on condition [0x16305000..0x16305e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 38 on 54310" daemon prio=10 tid=0x17372800 nid=0x3e2
waiting on condition [0x16356000..0x16356fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 37 on 54310" daemon prio=10 tid=0x17371800 nid=0x3e1
waiting on condition [0x163a7000..0x163a7f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 36 on 54310" daemon prio=10 tid=0x17025400 nid=0x3e0
waiting on condition [0x163f8000..0x163f90d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 35 on 54310" daemon prio=10 tid=0x17024000 nid=0x3df
waiting on condition [0x16449000..0x1644a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 34 on 54310" daemon prio=10 tid=0x17022c00 nid=0x3de
waiting on condition [0x1649b000..0x1649b1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 33 on 54310" daemon prio=10 tid=0x17021c00 nid=0x3dd
waiting on condition [0x164eb000..0x164ec150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 32 on 54310" daemon prio=10 tid=0x17020800 nid=0x3dc
waiting on condition [0x1653c000..0x1653ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 31 on 54310" daemon prio=10 tid=0x1701f800 nid=0x3db
waiting on condition [0x1658d000..0x1658de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 30 on 54310" daemon prio=10 tid=0x1701e800 nid=0x3da
waiting on condition [0x165de000..0x165defd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 29 on 54310" daemon prio=10 tid=0x1701d400 nid=0x3d9
waiting on condition [0x1662f000..0x1662ff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 28 on 54310" daemon prio=10 tid=0x1701c400 nid=0x3d8
waiting on condition [0x16680000..0x166810d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 27 on 54310" daemon prio=10 tid=0x1701b000 nid=0x3d7
waiting on condition [0x166d1000..0x166d2050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 26 on 54310" daemon prio=10 tid=0x1701a000 nid=0x3d6
waiting on condition [0x16723000..0x167231d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 25 on 54310" daemon prio=10 tid=0x17018c00 nid=0x3d5
waiting on condition [0x16773000..0x16774150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 24 on 54310" daemon prio=10 tid=0x17017c00 nid=0x3d4
waiting on condition [0x167c4000..0x167c4ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 23 on 54310" daemon prio=10 tid=0x17016800 nid=0x3d3
waiting on condition [0x16815000..0x16815e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 22 on 54310" daemon prio=10 tid=0x17015800 nid=0x3d2
waiting on condition [0x16866000..0x16866fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 21 on 54310" daemon prio=10 tid=0x17014400 nid=0x3d1
waiting on condition [0x168b7000..0x168b7f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 20 on 54310" daemon prio=10 tid=0x17013400 nid=0x3d0
waiting on condition [0x16908000..0x169090d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 19 on 54310" daemon prio=10 tid=0x17012000 nid=0x3cf
waiting on condition [0x16959000..0x1695a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 18 on 54310" daemon prio=10 tid=0x17011000 nid=0x3ce
waiting on condition [0x169ab000..0x169ab1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 17 on 54310" daemon prio=10 tid=0x1700fc00 nid=0x3cd
waiting on condition [0x169fb000..0x169fc150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 16 on 54310" daemon prio=10 tid=0x1700ec00 nid=0x3cc
waiting on condition [0x16a4c000..0x16a4ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 15 on 54310" daemon prio=10 tid=0x1700d800 nid=0x3cb
waiting on condition [0x16a9d000..0x16a9de50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 14 on 54310" daemon prio=10 tid=0x1700c800 nid=0x3ca
waiting on condition [0x16aee000..0x16aeefd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 13 on 54310" daemon prio=10 tid=0x1700b400 nid=0x3c9
waiting on condition [0x16b3f000..0x16b3ff50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 12 on 54310" daemon prio=10 tid=0x1700a400 nid=0x3c8
waiting on condition [0x16b90000..0x16b910d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 11 on 54310" daemon prio=10 tid=0x17009000 nid=0x3c7
waiting on condition [0x16be1000..0x16be2050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 10 on 54310" daemon prio=10 tid=0x17008400 nid=0x3c6
waiting on condition [0x16c33000..0x16c331d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 9 on 54310" daemon prio=10 tid=0x173d7000 nid=0x3c5
waiting on condition [0x16c83000..0x16c84150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 8 on 54310" daemon prio=10 tid=0x173d5c00 nid=0x3c4
waiting on condition [0x16cd4000..0x16cd4ed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 7 on 54310" daemon prio=10 tid=0x173fac00 nid=0x3c3
waiting on condition [0x16d25000..0x16d25e50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 6 on 54310" daemon prio=10 tid=0x173f9c00 nid=0x3c2
waiting on condition [0x16d76000..0x16d76fd0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 5 on 54310" daemon prio=10 tid=0x173fcc00 nid=0x3c1
waiting on condition [0x16dc7000..0x16dc7f50]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 4 on 54310" daemon prio=10 tid=0x17303800 nid=0x3c0
waiting on condition [0x16e18000..0x16e190d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 3 on 54310" daemon prio=10 tid=0x17324000 nid=0x3bf
waiting on condition [0x16e69000..0x16e6a050]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 2 on 54310" daemon prio=10 tid=0x1733d400 nid=0x3be
waiting on condition [0x16ebb000..0x16ebb1d0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 1 on 54310" daemon prio=10 tid=0x173ae800 nid=0x3bd
waiting on condition [0x16f0b000..0x16f0c150]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server handler 0 on 54310" daemon prio=10 tid=0x17311400 nid=0x3bc
waiting on condition [0x16f5c000..0x16f5ced0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1f7e2708> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:874)
Locked ownable synchronizers:
- None
"IPC Server listener on 54310" daemon prio=10 tid=0x173b1400 nid=0x3bb
waiting on condition [0x16fad000..0x16fade50]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.ipc.Server$Listener.run(Server.java:319)
Locked ownable synchronizers:
- None
"IPC Server Responder" daemon prio=10 tid=0x17305000 nid=0x3ba runnable
[0x16ffe000..0x16ffefd0]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x1f7e8b20> (a sun.nio.ch.Util$1)
- locked <0x1f7e8b10> (a java.util.Collections$UnmodifiableSet)
- locked <0x1f7e8928> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at org.apache.hadoop.ipc.Server$Responder.run(Server.java:455)
Locked ownable synchronizers:
- None
"SocketListener0-1" prio=10 tid=0x17308c00 nid=0x3b9 in Object.wait()
[0x1715c000..0x1715cf50]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x34d7b800> (a org.mortbay.util.ThreadPool$PoolThread)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:522)
- locked <0x34d7b800> (a org.mortbay.util.ThreadPool$PoolThread)
Locked ownable synchronizers:
- None
"Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=50070]"
prio=10 tid=0x1738b400 nid=0x3b7 runnable [0x1755c000..0x1755d050]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x34b77380> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.mortbay.util.ThreadedServer.acceptSocket(ThreadedServer.java:432)
at org.mortbay.util.ThreadedServer$Acceptor.run(ThreadedServer.java:631)
Locked ownable synchronizers:
- None
"SessionScavenger" daemon prio=10 tid=0x17309c00 nid=0x3b6 waiting on
condition [0x171ff000..0x171ff1d0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:587)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.FSNamesystem$DecommissionedMonitor@922804" daemon
prio=10 tid=0x09396c00 nid=0x3b5 waiting on condition
[0x175ae000..0x175ae150]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.dfs.FSNamesystem$DecommissionedMonitor.run(FSNamesystem.java:3602)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.FSNamesystem$ResolutionMonitor@7cd37a" daemon
prio=10 tid=0x09395800 nid=0x3b4 waiting on condition
[0x175fe000..0x175feed0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1fbfd5f8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.dfs.FSNamesystem$ResolutionMonitor.run(FSNamesystem.java:1840)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@1c4f0f8" daemon
prio=10 tid=0x09395000 nid=0x3b3 waiting on condition
[0x17769000..0x17769e50]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:2191)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.FSNamesystem$HeartbeatMonitor@129b0e1" daemon
prio=10 tid=0x08f64800 nid=0x3b1 waiting on condition
[0x1785c000..0x1785cf50]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.dfs.FSNamesystem$HeartbeatMonitor.run(FSNamesystem.java:2173)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"org.apache.hadoop.dfs.PendingReplicationBlocks$PendingReplicationMonitor@13f136e"
daemon prio=10 tid=0x08f64000 nid=0x3b0 waiting on condition
[0x1780b000..0x1780c0d0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.dfs.PendingReplicationBlocks$PendingReplicationMonitor.run(PendingReplicationBlocks.java:185)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"Timer thread for monitoring dfs" daemon prio=10 tid=0x09641400
nid=0x309 in Object.wait() [0x178ad000..0x178ade50]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1f7a9fc8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x1f7a9fc8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
Locked ownable synchronizers:
- None
"Timer thread for monitoring jvm" daemon prio=10 tid=0x0961e400
nid=0x308 in Object.wait() [0x178fe000..0x178fefd0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1f7b6f30> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x1f7b6f30> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
Locked ownable synchronizers:
- None
"RMI TCP Accept-0" daemon prio=10 tid=0x09607c00 nid=0x2d5 runnable
[0x17b1d000..0x17b1e0d0]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x1f78e4b0> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- None
"Low Memory Detector" daemon prio=10 tid=0x08a4cc00 nid=0x2be runnable
[0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"CompilerThread1" daemon prio=10 tid=0x08a4b400 nid=0x2bd waiting on
condition [0x00000000..0x19ac9698]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"CompilerThread0" daemon prio=10 tid=0x08a48c00 nid=0x2bc waiting on
condition [0x00000000..0x19b4a618]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Signal Dispatcher" daemon prio=10 tid=0x08a47800 nid=0x2bb runnable
[0x00000000..0x19b9bbb0]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Finalizer" daemon prio=10 tid=0x08a2c800 nid=0x2ba in Object.wait()
[0x19dec000..0x19dece50]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1f7effd0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x1f7effd0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
Locked ownable synchronizers:
- None
"Reference Handler" daemon prio=10 tid=0x08a2b400 nid=0x2b9 in
Object.wait() [0x19e3d000..0x19e3dfd0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1f7e6830> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x1f7e6830> (a java.lang.ref.Reference$Lock)
Locked ownable synchronizers:
- None
"main" prio=10 tid=0x08941c00 nid=0x2af in Object.wait()
[0xb7f14000..0xb7f142a8]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1f7bc1a0> (a org.apache.hadoop.ipc.RPC$Server)
at java.lang.Object.wait(Object.java:485)
at org.apache.hadoop.ipc.Server.join(Server.java:1021)
- locked <0x1f7bc1a0> (a org.apache.hadoop.ipc.RPC$Server)
at org.apache.hadoop.dfs.NameNode.join(NameNode.java:206)
at org.apache.hadoop.dfs.NameNode.main(NameNode.java:841)
Locked ownable synchronizers:
- None
"VM Thread" prio=10 tid=0x08a28400 nid=0x2b8 runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x08948400 nid=0x2b0 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x08949800 nid=0x2b1 runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x0894a800 nid=0x2b2 runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x0894b800 nid=0x2b3 runnable
"GC task thread#4 (ParallelGC)" prio=10 tid=0x0894c800 nid=0x2b4 runnable
"GC task thread#5 (ParallelGC)" prio=10 tid=0x0894dc00 nid=0x2b5 runnable
"GC task thread#6 (ParallelGC)" prio=10 tid=0x0894ec00 nid=0x2b6 runnable
"GC task thread#7 (ParallelGC)" prio=10 tid=0x0894fc00 nid=0x2b7 runnable
"VM Periodic Task Thread" prio=10 tid=0x09611000 nid=0x2d6 waiting on
condition
JNI global references: 1178
Raghu Angadi wrote:
>
> Give an example stack trace of a thread that is blocked on callQ and a
> handler thread that is not. trace of all the threads would be even
> better.
>
> If your handlers waiting on callQ, then they are waiting for work.
>
> Raghu.
>
> Sagar Naik wrote:
>> *Problem:*
>> The "ls" is taking noticeable time to respond.
>> *system:*
>> I have about 1.6 million files and namenode is prety much full with
>> heap(2400MB).
>>
>> I have configured dfs.handler.count to 100, and all IPC server
>> threads are locked onto a Queue (I think callQueue)
>>
>> In the namenode logs, I occasionally see Out of memory Exception.
>>
>> Any suggestions, to improve the response time of system ?
>> I can increase the dfs.handler.count to 256, but I guess I will see
>> more OOM exceptions.
>>
>
Re: Hadoop 18.1 "ls" stalls
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
Give an example stack trace of a thread that is blocked on callQ and a
handler thread that is not. trace of all the threads would be even better.
If your handlers waiting on callQ, then they are waiting for work.
Raghu.
Sagar Naik wrote:
> *Problem:*
> The "ls" is taking noticeable time to respond.
> *system:*
> I have about 1.6 million files and namenode is prety much full with
> heap(2400MB).
>
> I have configured dfs.handler.count to 100, and all IPC server threads
> are locked onto a Queue (I think callQueue)
>
> In the namenode logs, I occasionally see Out of memory Exception.
>
> Any suggestions, to improve the response time of system ?
> I can increase the dfs.handler.count to 256, but I guess I will see more
> OOM exceptions.
>