You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "nkeywal (Created) (JIRA)" <ji...@apache.org> on 2011/11/20 09:30:51 UTC
[jira] [Created] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
---------------------------------------------------------------------------------------
Key: HBASE-4832
URL: https://issues.apache.org/jira/browse/HBASE-4832
Project: HBase
Issue Type: Bug
Components: coprocessors, test
Affects Versions: 0.94.0
Reporter: nkeywal
Priority: Minor
The current implementation of HRegionServer#stop is
{noformat}
public void stop(final String msg) {
this.stopped = true;
LOG.info("STOPPED: " + msg);
synchronized (this) {
// Wakes run() if it is sleeping
notifyAll(); // FindBugs NN_NAKED_NOTIFY
}
}
{noformat}
The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
{noformat}
public void stop(final String msg) {
this.stopped = true;
LOG.info("STOPPED: " + msg);
// Wakes run() if it is sleeping
sleeper.skipSleepCycle();
}
{noformat}
Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
It likely because the code does no expect the region server to stop that fast.
The exception is:
{noformat}
testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
java.lang.Exception: test timed out after 30000 milliseconds
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.<init>(Throwable.java:196)
at java.lang.Exception.<init>(Exception.java:41)
at java.lang.InterruptedException.<init>(InterruptedException.java:48)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
{noformat}
We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated HBASE-4832:
---------------------------------
Attachment: HBASE-4832.patch
-Removes (timeout=30000) from @Test per nkeywal's suggestion.
-Add LOG.debug() concerning where interrupt occurs.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154933#comment-13154933 ]
nkeywal commented on HBASE-4832:
--------------------------------
One little comment: there is a conflict between the timeout on the method (@Test(timeout=timeout)) and the timeout of the sleep (Thread.sleep(timeout)). As they're both set to the same value (30 seconds), it can be one or another so the failure analysis will be more complex. I think we can remove the timeout on the method, the test itself ensures that it won't last forever.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated HBASE-4832:
---------------------------------
Attachment: HBASE-4832.patch
New version of the patch: parameterize test timeout (thanks to Ted Yu) and use this timeout amount in Thread.sleep() near end of testExceptionFromCoprocessorDuringPut().
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155574#comment-13155574 ]
Hadoop QA commented on HBASE-4832:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504809/HBASE-4832.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/339//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/339//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/339//console
This message is automatically generated.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157567#comment-13157567 ]
Hudson commented on HBASE-4832:
-------------------------------
Integrated in HBase-TRUNK #2485 (See [https://builds.apache.org/job/HBase-TRUNK/2485/])
HBASE-4832 TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated HBASE-4832:
---------------------------------
Attachment: HBASE-4832.patch
git diff --no-prefix
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154772#comment-13154772 ]
stack commented on HBASE-4832:
------------------------------
You good w/ this nkeywal? (Thanks Eugene for hacking on this)
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157685#comment-13157685 ]
Hudson commented on HBASE-4832:
-------------------------------
Integrated in HBase-TRUNK-security #11 (See [https://builds.apache.org/job/HBase-TRUNK-security/11/])
HBASE-4832 TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154825#comment-13154825 ]
Hadoop QA commented on HBASE-4832:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504678/HBASE-4832.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 65 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/323//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/323//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/323//console
This message is automatically generated.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155422#comment-13155422 ]
stack commented on HBASE-4832:
------------------------------
I can address the N comment on commit? (Removing the @Test timeout).
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated HBASE-4832:
---------------------------------
Attachment: HBASE-4832.patch
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832_trunk_hregionserver.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4832:
--------------------------
Attachment: 4832-timeout.txt
Patch which stores timeout value in a static variable.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154824#comment-13154824 ]
Hadoop QA commented on HBASE-4832:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504672/4832-timeout.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/322//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/322//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/322//console
This message is automatically generated.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157526#comment-13157526 ]
Ted Yu commented on HBASE-4832:
-------------------------------
Integrated to TRUNK.
Thanks for the patch, Eugene.
Thanks for the review Stack and N.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157612#comment-13157612 ]
Eugene Koontz commented on HBASE-4832:
--------------------------------------
Thanks to Ted for commit, Stack for reviews and nkeywal for filing and helping diagnose these problems with this test.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154764#comment-13154764 ]
Hadoop QA commented on HBASE-4832:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504581/HBASE-4832.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 65 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/321//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/321//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/321//console
This message is automatically generated.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4832:
--------------------------
Fix Version/s: 0.94.0
Release Note: (was: This incorporates nkeywal's earlier patch to this JIRA, and allows TestRegionServerCoprocessortWithAbort() to work with it. It changes the test to use a Zookeeper watcher in a separate thread to watch for the regionserver to abort. (This is also what is currently done with TestMasterCoprocessorWithAbort()).
In my testing, repeated iterations (30+) of TestRegionServerCoprocessortWithAbort() succeed.)
Hadoop Flags: Reviewed
This incorporates nkeywal's earlier patch to this JIRA, and allows TestRegionServerCoprocessortWithAbort() to work with it. It changes the test to use a Zookeeper watcher in a separate thread to watch for the regionserver to abort. (This is also what is currently done with TestMasterCoprocessorWithAbort()).
In Eugene's testing, repeated iterations (30+) of TestRegionServerCoprocessortWithAbort() succeed.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154719#comment-13154719 ]
Ted Yu commented on HBASE-4832:
-------------------------------
+1 on patch.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832_trunk_hregionserver.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4832:
--------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155466#comment-13155466 ]
Eugene Koontz commented on HBASE-4832:
--------------------------------------
@stack, that is fine, thanks.
-Eugene
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz reassigned HBASE-4832:
------------------------------------
Assignee: Eugene Koontz
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832_trunk_hregionserver.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155453#comment-13155453 ]
stack commented on HBASE-4832:
------------------------------
And N, you want to uncomment this section now? This patch wants to do it.
{code}
public void stop(final String msg) {
this.stopped = true;
LOG.info("STOPPED: " + msg);
// Wakes run() if it is sleeping
//sleeper.skipSleepCycle();
//will be uncommented later, see discussion in jira 4798
}
{code}
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154766#comment-13154766 ]
Ted Yu commented on HBASE-4832:
-------------------------------
The test failures were due to 'Too many open files'
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157476#comment-13157476 ]
nkeywal commented on HBASE-4832:
--------------------------------
I believe this patch could be integrated?
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4832:
---------------------------
Attachment: 4832_trunk_hregionserver.patch
4832_trunk_hregionserver.patch contains the fix on HRegionServer which makes the coprocessor test fails.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Priority: Minor
> Attachments: 4832_trunk_hregionserver.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155570#comment-13155570 ]
nkeywal commented on HBASE-4832:
--------------------------------
fyi, the patch for the region server itself is in HBASE-4833, if the trunk changed I will update the patch.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated HBASE-4832:
---------------------------------
Release Note:
This incorporates nkeywal's earlier patch to this JIRA, and allows TestRegionServerCoprocessortWithAbort() to work with it. It changes the test to use a Zookeeper watcher in a separate thread to watch for the regionserver to abort. (This is also what is currently done with TestMasterCoprocessorWithAbort()).
In my testing, repeated iterations (30+) of TestRegionServerCoprocessortWithAbort() succeed.
Status: Patch Available (was: Open)
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832_trunk_hregionserver.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4832)
TestRegionServerCoprocessorExceptionWithAbort fails if the region server
stops too fast
Posted by "Eugene Koontz (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155468#comment-13155468 ]
Eugene Koontz commented on HBASE-4832:
--------------------------------------
@stack, I tried with "sleeper.skipSleepCycle()" uncommented and commented; test consistently succeeded 30+ iterations in both cases.
> TestRegionServerCoprocessorExceptionWithAbort fails if the region server stops too fast
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-4832
> URL: https://issues.apache.org/jira/browse/HBASE-4832
> Project: HBase
> Issue Type: Bug
> Components: coprocessors, test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: 4832-timeout.txt, 4832_trunk_hregionserver.patch, HBASE-4832.patch, HBASE-4832.patch
>
>
> The current implementation of HRegionServer#stop is
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> synchronized (this) {
> // Wakes run() if it is sleeping
> notifyAll(); // FindBugs NN_NAKED_NOTIFY
> }
> }
> {noformat}
> The notification is sent on the wrong object and does nothing. As a consequence, the region server continues to sleep instead of waking up and stopping immediately. A correct implementation is:
> {noformat}
> public void stop(final String msg) {
> this.stopped = true;
> LOG.info("STOPPED: " + msg);
> // Wakes run() if it is sleeping
> sleeper.skipSleepCycle();
> }
> {noformat}
> Then the region server stops immediately. This makes the region server stops 0,5s faster on average, which is quite useful for unit tests.
> However, with this fix, TestRegionServerCoprocessorExceptionWithAbort does not work.
> It likely because the code does no expect the region server to stop that fast.
> The exception is:
> {noformat}
> testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort) Time elapsed: 30.06 sec <<< ERROR!
> java.lang.Exception: test timed out after 30000 milliseconds
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.<init>(Throwable.java:196)
> at java.lang.Exception.<init>(Exception.java:41)
> at java.lang.InterruptedException.<init>(InterruptedException.java:48)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:804)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:778)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:697)
> at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:75)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1280)
> at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:585)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:154)
> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:357)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:866)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:920)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:808)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1469)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1354)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:892)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:750)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:725)
> at org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.testExceptionFromCoprocessorDuringPut(TestRegionServerCoprocessorExceptionWithAbort.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> {noformat}
> We have this exception because we entered a loop of retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira