You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Roshan Naik <ro...@hortonworks.com> on 2013/11/07 04:04:35 UTC
Asynch Hbase sink hanging with asynchbase 0.
The AsyncHbase 1.5.0 Library is currently in RC2 phase. it should help
fix the protocol incompat issues introduced in Hbase 0.96.
I was tyring out the async hbase sink with it and hbase-0.96. The
TestAsyncHBaseSink hangs. Below is the Jstack output on the hung unit test:
Any thoughts ?
"main" prio=5 tid=7f8fd3001800 nid=0x10c08a000 waiting on condition
[10c087000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <7f434d840> (a
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
at
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
at
org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:440)
at
org.apache.flume.sink.hbase.TestAsyncHBaseSink.testTimeOut(TestAsyncHBaseSink.java:212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:22)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Asynch Hbase sink hanging with asynchbase 0.
Posted by Hari Shreedharan <hs...@cloudera.com>.
Looks like the hang is in the callback for tableExists(). Note sure if that is something caused by API issues. Maybe bring it up with the asynchbase folks?
Thanks,
Hari
On Wednesday, November 6, 2013 at 7:42 PM, Roshan Naik wrote:
> fyi .. about the asynchbase RC2 ...
>
> https://groups.google.com/forum/#!topic/asynchbase/HFiG3-e4M_0
>
>
> On Wed, Nov 6, 2013 at 7:04 PM, Roshan Naik <roshan@hortonworks.com (mailto:roshan@hortonworks.com)> wrote:
>
> > The AsyncHbase 1.5.0 Library is currently in RC2 phase. it should help
> > fix the protocol incompat issues introduced in Hbase 0.96.
> >
> > I was tyring out the async hbase sink with it and hbase-0.96. The
> > TestAsyncHBaseSink hangs. Below is the Jstack output on the hung unit test:
> >
> > Any thoughts ?
> >
> >
> > "main" prio=5 tid=7f8fd3001800 nid=0x10c08a000 waiting on condition
> > [10c087000]
> >
> > java.lang.Thread.State: WAITING (parking)
> >
> > at sun.misc.Unsafe.park(Native Method)
> >
> > - parking to wait for <7f434d840> (a
> > java.util.concurrent.CountDownLatch$Sync)
> >
> > at
> > java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
> >
> > at
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
> >
> > at
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
> >
> > at
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
> >
> > at
> > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
> >
> > at
> > org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:440)
> >
> > at
> > org.apache.flume.sink.hbase.TestAsyncHBaseSink.testTimeOut(TestAsyncHBaseSink.java:212)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >
> > at
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> >
> > at
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> >
> > at
> > org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:22)
> >
> > at
> > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> >
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> >
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> >
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> >
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> >
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> >
> > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> >
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> >
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> >
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> >
> > at
> > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> >
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> >
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> >
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> >
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> >
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> >
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> >
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> >
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
> >
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
Re: Asynch Hbase sink hanging with asynchbase 0.
Posted by Roshan Naik <ro...@hortonworks.com>.
fyi .. about the asynchbase RC2 ...
https://groups.google.com/forum/#!topic/asynchbase/HFiG3-e4M_0
On Wed, Nov 6, 2013 at 7:04 PM, Roshan Naik <ro...@hortonworks.com> wrote:
> The AsyncHbase 1.5.0 Library is currently in RC2 phase. it should help
> fix the protocol incompat issues introduced in Hbase 0.96.
>
> I was tyring out the async hbase sink with it and hbase-0.96. The
> TestAsyncHBaseSink hangs. Below is the Jstack output on the hung unit test:
>
> Any thoughts ?
>
>
> "main" prio=5 tid=7f8fd3001800 nid=0x10c08a000 waiting on condition
> [10c087000]
>
> java.lang.Thread.State: WAITING (parking)
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <7f434d840> (a
> java.util.concurrent.CountDownLatch$Sync)
>
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>
> at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
>
> at
> org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:440)
>
> at
> org.apache.flume.sink.hbase.TestAsyncHBaseSink.testTimeOut(TestAsyncHBaseSink.java:212)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>
> at
> org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:22)
>
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>
> at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
>
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
>
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
>
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
>
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
>
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
>
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.