You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Mohit Anchlia <mo...@gmail.com> on 2012/08/22 21:54:30 UTC
Application hanging writing to Flume HDFS sink
I have 5 threads writing json blob to hdfs but what I see is application
hanging and when I do the thread dump on the application I see threads
waiting on epollWait:
"New I/O client worker #5-1" daemon prio=10 tid=0x00002aaac81e3000
nid=0x7080 runnable [0x000000004395c000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x000000060c3cc170> (a sun.nio.ch.Util$2)
- locked <0x000000060c3cc160> (a
java.util.Collections$UnmodifiableSet)
- locked <0x000000060c3cbf58> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:38)
at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:165)
at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Re: Application hanging writing to Flume HDFS sink
Posted by Hari Shreedharan <hs...@cloudera.com>.
Why do you say they are waiting? They are in Runnable state. That said this seems to be due to this: https://github.com/netty/netty/issues/327
I am not sure if this is the same issue, but 100% CPU can be caused by the above.
Hari
--
Hari Shreedharan
On Wednesday, August 22, 2012 at 12:54 PM, Mohit Anchlia wrote:
> I have 5 threads writing json blob to hdfs but what I see is application hanging and when I do the thread dump on the application I see threads waiting on epollWait:
>
> "New I/O client worker #5-1" daemon prio=10 tid=0x00002aaac81e3000 nid=0x7080 runnable [0x000000004395c000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x000000060c3cc170> (a sun.nio.ch.Util$2)
> - locked <0x000000060c3cc160> (a java.util.Collections$UnmodifiableSet)
> - locked <0x000000060c3cbf58> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:38)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:165)
> at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)