You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "Christopher Tubbs (JIRA)" <ji...@apache.org> on 2016/08/26 22:40:20 UTC
[jira] [Comment Edited] (ACCUMULO-4425)
VolumeIT.testDirtyReplaceVolumes fails
[ https://issues.apache.org/jira/browse/ACCUMULO-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440073#comment-15440073 ]
Christopher Tubbs edited comment on ACCUMULO-4425 at 8/26/16 10:39 PM:
-----------------------------------------------------------------------
It looks like when it fails, it routinely fails because there's still a CLOSED WAL still referenced. When I wrap this check with some code to continuously retry, I get the following failure:
{code}
org.apache.accumulo.server.log.WalStateManager$WalMarkerException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /accumulo/28c6cf19-0475-4366-9ded-65b14376f893/wals/myhostname:39999[156c8e5b31f0003]/22d2b5a5-181a-404c-8dcb-51bf3aace2f6
at org.apache.accumulo.server.log.WalStateManager.state(WalStateManager.java:199)
at org.apache.accumulo.server.log.WalStateManager.getAllState(WalStateManager.java:208)
at org.apache.accumulo.test.VolumeIT.verifyVolumesUsed(VolumeIT.java:436)
at org.apache.accumulo.test.VolumeIT.testReplaceVolume(VolumeIT.java:537)
at org.apache.accumulo.test.VolumeIT.testDirtyReplaceVolumes(VolumeIT.java:570)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /accumulo/28c6cf19-0475-4366-9ded-65b14376f893/wals/myhostname:39999[156c8e5b31f0003]/22d2b5a5-181a-404c-8dcb-51bf3aace2f6
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184)
at org.apache.accumulo.fate.zookeeper.ZooReader.getData(ZooReader.java:75)
at org.apache.accumulo.fate.zookeeper.ZooReader.getData(ZooReader.java:67)
at org.apache.accumulo.server.log.WalStateManager.state(WalStateManager.java:197)
... 18 more
{code}
It looks like the {{WalStateManager}} doesn't like when the WAL state is deleted from underneath it.
was (Author: ctubbsii):
It looks like when it fails, it routinely fails because there's still a CLOSED WAL still referenced. When I wrap this check with some code to continuously retry, I get the following failure:
{code}
org.apache.accumulo.server.log.WalStateManager$WalMarkerException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /accumulo/28c6cf19-0475-4366-9ded-65b14376f893/wals/myhostname:39999[156c8e5b31f0003]/22d2b5a5-181a-404c-8dcb-51bf3aace2f6
at org.apache.accumulo.server.log.WalStateManager.state(WalStateManager.java:199)
at org.apache.accumulo.server.log.WalStateManager.getAllState(WalStateManager.java:208)
at org.apache.accumulo.test.VolumeIT.verifyVolumesUsed(VolumeIT.java:436)
at org.apache.accumulo.test.VolumeIT.testReplaceVolume(VolumeIT.java:537)
at org.apache.accumulo.test.VolumeIT.testDirtyReplaceVolumes(VolumeIT.java:570)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /accumulo/28c6cf19-0475-4366-9ded-65b14376f893/wals/canx2ul-myhostname:39999[156c8e5b31f0003]/22d2b5a5-181a-404c-8dcb-51bf3aace2f6
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184)
at org.apache.accumulo.fate.zookeeper.ZooReader.getData(ZooReader.java:75)
at org.apache.accumulo.fate.zookeeper.ZooReader.getData(ZooReader.java:67)
at org.apache.accumulo.server.log.WalStateManager.state(WalStateManager.java:197)
... 18 more
{code}
It looks like the {{WalStateManager}} doesn't like when the WAL state is deleted from underneath it.
> VolumeIT.testDirtyReplaceVolumes fails
> --------------------------------------
>
> Key: ACCUMULO-4425
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4425
> Project: Accumulo
> Issue Type: Bug
> Reporter: Christopher Tubbs
> Priority: Blocker
> Fix For: 1.8.0
>
>
> *Error Message*
> {code}
> Unexpected volume file:/var/lib/jenkins/workspace/Accumulo-1.8-ITs-failures/test/target/mini-tests/org.apache.accumulo.test.VolumeIT_testDirtyReplaceVolumes/volumes/v1/wal/jenkins.revelc.net+38766/3eb39803-c014-4195-943a-7a12efa2f515
> {code}
> *Stacktrace*
> {code}
> java.lang.AssertionError: Unexpected volume file:/var/lib/jenkins/workspace/Accumulo-1.8-ITs-failures/test/target/mini-tests/org.apache.accumulo.test.VolumeIT_testDirtyReplaceVolumes/volumes/v1/wal/jenkins.revelc.net+38766/3eb39803-c014-4195-943a-7a12efa2f515
> at org.apache.accumulo.test.VolumeIT.verifyVolumesUsed(VolumeIT.java:441)
> at org.apache.accumulo.test.VolumeIT.testReplaceVolume(VolumeIT.java:533)
> at org.apache.accumulo.test.VolumeIT.testDirtyReplaceVolumes(VolumeIT.java:566)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)