You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "nkeywal (Created) (JIRA)" <ji...@apache.org> on 2011/10/30 19:30:32 UTC
[jira] [Created] (HBASE-4703) Improvements in tests
Improvements in tests
---------------------
Key: HBASE-4703
URL: https://issues.apache.org/jira/browse/HBASE-4703
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Global:
- when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
- 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
- tmp dir management: to allow multiple tests using dir to run on the same machine without conflicting, the tmp dir is set once per JVM. This should fix HBASE-3672 as well, because the map reduce cluster is configured to use the hadoop tmp dir
- Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
Local issues:
- TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
- TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
- TestMemoryBoundedLogMessageBuffer useless System.out.println
- io.hfile.TestReseekTo useless System.out.println
- TestTableInputFormat does not shutdown the cluster
- testGlobalMemStore does not shutdown the cluster
- rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
- HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
- TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141698#comment-13141698 ]
nkeywal commented on HBASE-4703:
--------------------------------
It seems that it finally worked in the last trunk... and this proves nothing unfortunately.
I compared the dates in the logs. There are just 15 minutes between the two tests around TestAdmin, it shows that it's the timeout in the pom.xml that took place. I would propose to add @Test(timeout=xxx) for all tests in TestAdmin, that could help to identify what is the issue. I can do that in this JIRA or in another one.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140306#comment-13140306 ]
Hadoop QA commented on HBASE-4703:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501620/20111030_4703_all.v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 177 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/111//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/111//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/111//console
This message is automatically generated.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141762#comment-13141762 ]
nkeywal commented on HBASE-4703:
--------------------------------
When I tried my patch, I reproduced on the trunk the 15minutes timeout. Adding a timeout on each method does not help, but I have the traces.
First, I have this, may be it's an issue in my env. I've just pulled the trunk.
{noformat}
2011-11-01 15:48:40,744 WARN [Master:0;localhost,39664,1320187706355] master.AssignmentManager(1471): Failed assignment of -ROOT-,,0.70236052 to localhost,44046,1320187706849, trying to assign elsewhere instead; retry=1
org.apache.hadoop.hbase.ipc.HBaseRPC$VersionMismatch: Protocol org.apache.hadoop.hbase.ipc.HRegionInterface version mismatch. (client = 28, server = 29)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:185)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:300)
{noformat}
Anyway, after this the logs finishes with:
{noformat}
2011-11-01 15:54:35,132 INFO [Master:0;localhost,39664,1320187706355.oldLogCleaner] hbase.Chore(80): Master:0;localhost,39664,1320187706355.oldLogCleaner exiting
Process Thread Dump: Automatic Stack Trace every 60 seconds waiting on Master:0;localhost,39664,1320187706355
{noformat}
it's in
{noformat}
sun.management.ThreadImpl.getThreadInfo1(Native Method)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:156)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:121)
org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:149)
org.apache.hadoop.hbase.util.Threads.threadDumpingIsAlive(Threads.java:113)
org.apache.hadoop.hbase.LocalHBaseCluster.join(LocalHBaseCluster.java:405)
org.apache.hadoop.hbase.MiniHBaseCluster.join(MiniHBaseCluster.java:408)
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniHBaseCluster(HBaseTestingUtility.java:616)
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniCluster(HBaseTestingUtility.java:590)
org.apache.hadoop.hbase.client.TestAdmin.tearDownAfterClass(TestAdmin.java:89)
{noformat}
So that's at least why adding a timeout wont help and may be why it does not end at all. Adding a maximum retry to Threads#threadDumpingIsAlive could help.
I also wonder if the root cause of the non ending is my modif on the wal, with some threads surprised to have updates that were not written in the wal. Here is the full stack dump:
{noformat}
Thread 354 (IPC Client (47) connection to localhost/127.0.0.1:52227 from nkeywal):
State: TIMED_WAITING
Blocked count: 360
Waited count: 359
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:702)
org.apache.hadoop.ipc.Client$Connection.run(Client.java:744)
Thread 272 (Master:0;localhost,39664,1320187706355-EventThread):
State: WAITING
Blocked count: 0
Waited count: 4
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@107b954b
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
Thread 271 (Master:0;localhost,39664,1320187706355-SendThread(localhost:21819)):
State: RUNNABLE
Blocked count: 2
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1107)
Thread 152 (Master:0;localhost,39664,1320187706355):
State: WAITING
Blocked count: 217
Waited count: 174
Waiting on org.apache.hadoop.hbase.zookeeper.RootRegionTracker@6621477c
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:131)
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:104)
org.apache.hadoop.hbase.catalog.CatalogTracker.waitForRoot(CatalogTracker.java:277)
org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:523)
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
java.lang.Thread.run(Thread.java:662)
Thread 165 (LruBlockCache.EvictionThread):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread@3e9d7b56
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:593)
java.lang.Thread.run(Thread.java:662)
Thread 151 (main-EventThread):
State: WAITING
Blocked count: 40
Waited count: 39
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@22531380
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
Thread 150 (main-SendThread(localhost:21819)):
State: RUNNABLE
Blocked count: 55
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1107)
Thread 149 (IPC Server handler 4 on 39664):
State: WAITING
Blocked count: 3
Waited count: 1968
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73f5173f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1224)
Thread 148 (IPC Server handler 3 on 39664):
State: WAITING
Blocked count: 7
Waited count: 1969
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73f5173f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1224)
Thread 147 (IPC Server handler 2 on 39664):
State: WAITING
Blocked count: 11
Waited count: 1968
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73f5173f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1224)
Thread 146 (IPC Server handler 1 on 39664):
State: WAITING
Blocked count: 6
Waited count: 1969
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73f5173f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1224)
Thread 145 (IPC Server handler 0 on 39664):
State: WAITING
Blocked count: 3
Waited count: 1970
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73f5173f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1224)
Thread 133 (IPC Server listener on 39664):
State: RUNNABLE
Blocked count: 1
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:580)
Thread 144 (IPC Server Responder):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.hadoop.hbase.ipc.HBaseServer$Responder.run(HBaseServer.java:754)
Thread 143 (IPC Reader 9 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 142 (IPC Reader 8 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 141 (IPC Reader 7 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 140 (IPC Reader 6 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 139 (IPC Reader 5 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 138 (IPC Reader 4 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 137 (IPC Reader 3 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 136 (IPC Reader 2 on port 39664):
State: RUNNABLE
Blocked count: 1
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 135 (IPC Reader 1 on port 39664):
State: RUNNABLE
Blocked count: 1
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 134 (IPC Reader 0 on port 39664):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:471)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 124 (LeaseChecker):
State: TIMED_WAITING
Blocked count: 0
Waited count: 407
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1252)
java.lang.Thread.run(Thread.java:662)
Thread 121 (ProcessThread:-1):
State: WAITING
Blocked count: 0
Waited count: 269
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f5d84e0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:103)
Thread 120 (SyncThread:0):
State: WAITING
Blocked count: 2
Waited count: 252
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3b25b27c
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:94)
Thread 119 (SessionTracker):
State: TIMED_WAITING
Blocked count: 0
Waited count: 207
Stack:
java.lang.Object.wait(Native Method)
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:142)
Thread 118 (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:21819):
State: RUNNABLE
Blocked count: 38
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:232)
Thread 116 (org.apache.hadoop.hdfs.server.datanode.DataBlockScanner@2e6f947b):
State: TIMED_WAITING
Blocked count: 0
Waited count: 407
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:620)
java.lang.Thread.run(Thread.java:662)
Thread 113 (IPC Server handler 2 on 39899):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1b4c72c8
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 112 (IPC Server handler 1 on 39899):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1b4c72c8
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 111 (IPC Server handler 0 on 39899):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1b4c72c8
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 106 (IPC Server listener on 39899):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener.run(Server.java:439)
Thread 108 (IPC Server Responder):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.hadoop.ipc.Server$Responder.run(Server.java:605)
Thread 102 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@45d1c3cd):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:152)
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:131)
java.lang.Thread.run(Thread.java:662)
Thread 109 (DataNode: [/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data5,/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data6]):
State: TIMED_WAITING
Blocked count: 137
Waited count: 283
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataNode.offerService(DataNode.java:986)
org.apache.hadoop.hdfs.server.datanode.DataNode.run(DataNode.java:1421)
java.lang.Thread.run(Thread.java:662)
Thread 107 (pool-4-thread-1):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:333)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 105 (Timer-3):
State: TIMED_WAITING
Blocked count: 0
Waited count: 15
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:509)
java.util.TimerThread.run(Timer.java:462)
Thread 104 (1353906974@qtp-1197660496-1 - Acceptor0 SelectChannelConnector@localhost:59858):
State: RUNNABLE
Blocked count: 1
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Thread 103 (1185716172@qtp-1197660496-0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 8
Stack:
java.lang.Object.wait(Native Method)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
Thread 101 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data6):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 98 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data5):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 90 (org.apache.hadoop.hdfs.server.datanode.DataBlockScanner@605b28c9):
State: TIMED_WAITING
Blocked count: 0
Waited count: 408
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:620)
java.lang.Thread.run(Thread.java:662)
Thread 86 (IPC Server handler 2 on 56098):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78ff22ed
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 85 (IPC Server handler 1 on 56098):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78ff22ed
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 84 (IPC Server handler 0 on 56098):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78ff22ed
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 80 (IPC Server listener on 56098):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener.run(Server.java:439)
Thread 82 (IPC Server Responder):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.hadoop.ipc.Server$Responder.run(Server.java:605)
Thread 76 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@2b68989e):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:152)
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:131)
java.lang.Thread.run(Thread.java:662)
Thread 83 (DataNode: [/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data3,/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data4]):
State: TIMED_WAITING
Blocked count: 149
Waited count: 290
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataNode.offerService(DataNode.java:986)
org.apache.hadoop.hdfs.server.datanode.DataNode.run(DataNode.java:1421)
java.lang.Thread.run(Thread.java:662)
Thread 81 (pool-3-thread-1):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:333)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 79 (Timer-2):
State: TIMED_WAITING
Blocked count: 0
Waited count: 15
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:509)
java.util.TimerThread.run(Timer.java:462)
Thread 78 (348878159@qtp-164967086-1 - Acceptor0 SelectChannelConnector@localhost:36427):
State: RUNNABLE
Blocked count: 2
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Thread 77 (1905729203@qtp-164967086-0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 8
Stack:
java.lang.Object.wait(Native Method)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
Thread 75 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data4):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 72 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data3):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 61 (org.apache.hadoop.hdfs.server.datanode.DataBlockScanner@6709da93):
State: TIMED_WAITING
Blocked count: 0
Waited count: 408
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:620)
java.lang.Thread.run(Thread.java:662)
Thread 60 (IPC Server handler 2 on 40388):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56afd9e3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 59 (IPC Server handler 1 on 40388):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56afd9e3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 58 (IPC Server handler 0 on 40388):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56afd9e3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 53 (IPC Server listener on 40388):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener.run(Server.java:439)
Thread 55 (IPC Server Responder):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.hadoop.ipc.Server$Responder.run(Server.java:605)
Thread 49 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@48baa31b):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:152)
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:131)
java.lang.Thread.run(Thread.java:662)
Thread 57 (DataNode: [/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data1,/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data2]):
State: TIMED_WAITING
Blocked count: 148
Waited count: 283
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hdfs.server.datanode.DataNode.offerService(DataNode.java:986)
org.apache.hadoop.hdfs.server.datanode.DataNode.run(DataNode.java:1421)
java.lang.Thread.run(Thread.java:662)
Thread 54 (pool-2-thread-1):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:333)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 52 (Timer-1):
State: TIMED_WAITING
Blocked count: 0
Waited count: 15
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:509)
java.util.TimerThread.run(Timer.java:462)
Thread 51 (1031841975@qtp-1068922666-1 - Acceptor0 SelectChannelConnector@localhost:56535):
State: RUNNABLE
Blocked count: 1
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Thread 50 (1191055321@qtp-1068922666-0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 8
Stack:
java.lang.Object.wait(Native Method)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
Thread 48 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data2):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 45 (refreshUsed-/tmp/d846d12d-19f3-439a-8cab-e05db67ae60b/dfscluster_2773516a-bc5a-4029-9ee0-fc499eee6ca8/dfs/data/data1):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:80)
java.lang.Thread.run(Thread.java:662)
Thread 31 (IPC Server handler 9 on 52227):
State: WAITING
Blocked count: 0
Waited count: 64
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 30 (IPC Server handler 8 on 52227):
State: WAITING
Blocked count: 3
Waited count: 65
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 29 (IPC Server handler 7 on 52227):
State: WAITING
Blocked count: 2
Waited count: 66
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 28 (IPC Server handler 6 on 52227):
State: WAITING
Blocked count: 1
Waited count: 64
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 27 (IPC Server handler 5 on 52227):
State: WAITING
Blocked count: 4
Waited count: 68
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 26 (IPC Server handler 4 on 52227):
State: WAITING
Blocked count: 0
Waited count: 64
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 25 (IPC Server handler 3 on 52227):
State: WAITING
Blocked count: 0
Waited count: 65
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 24 (IPC Server handler 2 on 52227):
State: WAITING
Blocked count: 1
Waited count: 66
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 23 (IPC Server handler 1 on 52227):
State: WAITING
Blocked count: 0
Waited count: 64
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 22 (IPC Server handler 0 on 52227):
State: WAITING
Blocked count: 3
Waited count: 65
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64473a14
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:1364)
Thread 15 (IPC Server listener on 52227):
State: RUNNABLE
Blocked count: 43
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener.run(Server.java:439)
Thread 17 (IPC Server Responder):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.apache.hadoop.ipc.Server$Responder.run(Server.java:605)
Thread 21 (Timer-0):
State: TIMED_WAITING
Blocked count: 1
Waited count: 17
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:509)
java.util.TimerThread.run(Timer.java:462)
Thread 20 (245022965@qtp-589734556-1 - Acceptor0 SelectChannelConnector@localhost:33079):
State: RUNNABLE
Blocked count: 2
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Thread 19 (1821050251@qtp-589734556-0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 8
Stack:
java.lang.Object.wait(Native Method)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
Thread 16 (pool-1-thread-1):
State: RUNNABLE
Blocked count: 45
Waited count: 44
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:333)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)
Thread 14 (org.apache.hadoop.hdfs.server.namenode.DecommissionManager$Monitor@620fa83):
State: TIMED_WAITING
Blocked count: 1
Waited count: 147
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.namenode.DecommissionManager$Monitor.run(DecommissionManager.java:65)
java.lang.Thread.run(Thread.java:662)
Thread 13 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$ReplicationMonitor@d5d4de6):
State: TIMED_WAITING
Blocked count: 2
Waited count: 147
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:2689)
java.lang.Thread.run(Thread.java:662)
Thread 12 (org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@73d32e45):
State: TIMED_WAITING
Blocked count: 0
Waited count: 220
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:368)
java.lang.Thread.run(Thread.java:662)
Thread 11 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$HeartbeatMonitor@19be4777):
State: TIMED_WAITING
Blocked count: 0
Waited count: 89
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem$HeartbeatMonitor.run(FSNamesystem.java:2666)
java.lang.Thread.run(Thread.java:662)
Thread 10 (org.apache.hadoop.hdfs.server.namenode.PendingReplicationBlocks$PendingReplicationMonitor@658f7386):
State: TIMED_WAITING
Blocked count: 0
Waited count: 2
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.server.namenode.PendingReplicationBlocks$PendingReplicationMonitor.run(PendingReplicationBlocks.java:186)
java.lang.Thread.run(Thread.java:662)
Thread 4 (Signal Dispatcher):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
Thread 3 (Finalizer):
State: WAITING
Blocked count: 9
Waited count: 10
Waiting on java.lang.ref.ReferenceQueue$Lock@3d1a70a7
Stack:
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
Thread 2 (Reference Handler):
State: WAITING
Blocked count: 11
Waited count: 12
Waiting on java.lang.ref.Reference$Lock@270d75a3
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
Thread 1 (main):
State: RUNNABLE
Blocked count: 104
Waited count: 97
Stack:
sun.management.ThreadImpl.getThreadInfo1(Native Method)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:156)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:121)
org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:149)
org.apache.hadoop.hbase.util.Threads.threadDumpingIsAlive(Threads.java:113)
org.apache.hadoop.hbase.LocalHBaseCluster.join(LocalHBaseCluster.java:405)
org.apache.hadoop.hbase.MiniHBaseCluster.join(MiniHBaseCluster.java:408)
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniHBaseCluster(HBaseTestingUtility.java:616)
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniCluster(HBaseTestingUtility.java:590)
org.apache.hadoop.hbase.client.TestAdmin.tearDownAfterClass(TestAdmin.java:89)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
org.junit.runners.ParentRunner.run(ParentRunner.java:236)
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
{noformat}
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Patch Available (was: Open)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141705#comment-13141705 ]
stack commented on HBASE-4703:
------------------------------
Want to do this in a new issue N?
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139730#comment-13139730 ]
Ted Yu commented on HBASE-4703:
-------------------------------
Currently patch testing uses -p0 to apply patches.
>From Todd:
Try "git show --no-prefix" or "git diff --no-prefix" instead. That way it will apply with patch -p0 and should work in test-patch
Also, the observations in this JIRA should be broadcast and receive wider discussion so that we can establish well-accepted guidelines.
Otherwise the issues corrected by this effort would get back into hbase codebase.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140210#comment-13140210 ]
nkeywal commented on HBASE-4703:
--------------------------------
I've managed to reproduce the stack on testRowRange(org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol; after more than 10 tries. I am going to revert my change on this file. I haven't been able to reproduce the others (two are caused by a java.io.IOException: Too many open files).
I had a stack on testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting), but it's a stack I've already seen without this patch, so I don't think it's related.
{noformat}
testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting) Time elapsed: 231.598 sec <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at org.junit.Assert.assertEquals(Assert.java:454)
at org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testWorkerAbort(TestDistributedLogSplitting.java:294)
{noformat}
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141957#comment-13141957 ]
nkeywal commented on HBASE-4703:
--------------------------------
Yes, the join are failing, and as the main thread does not end the JVM does not end. I pulled the trunk again, I still have the issue on rpc version.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Attachment: 20111030_4703_all.v3.patch
Should fix the points above.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141772#comment-13141772 ]
stack commented on HBASE-4703:
------------------------------
If '(client = 28, server = 29)' is not an issue in your environment, some rpc version setting got messed up (Trying it here).
Are you saying that we are stuck at:
{code}
...
org.apache.hadoop.hbase.util.Threads.threadDumpingIsAlive(Threads.java:113)
{code}
... and we never let go? That'd be be strange (would explain why builds hang on jenkins sometimes though). Isn't this joined on another thread? Whats the other thread doing?
That stack trace is horrid to look at. Can you make sense of it? Whats it stuck on?
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Patch Available (was: Open)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140007#comment-13140007 ]
Hadoop QA commented on HBASE-4703:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501578/20111030_4703_all.v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 180 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/109//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/109//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/109//console
This message is automatically generated.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141006#comment-13141006 ]
Hudson commented on HBASE-4703:
-------------------------------
Integrated in HBase-0.92 #92 (See [https://builds.apache.org/job/HBase-0.92/92/])
HBASE-4703 Improvements in tests
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TimestampTestBase.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMultipleTimestamps.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestScannerTimeout.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestTimestampsFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestAggregateProtocol.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpoint.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestColumnRangeFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TsvImporterCustomTestMapper.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/monitoring/TestMemoryBoundedLogMessageBuffer.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestGzipFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestScannersWithFilters.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestSchemaResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestTransform.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestVersionResource.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteAdmin.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteTable.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestIdLock.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139755#comment-13139755 ]
nkeywal commented on HBASE-4703:
--------------------------------
Yes, I was thinking about writing a small doc, I will have a look at the
existing section and propose some modifications.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140343#comment-13140343 ]
stack commented on HBASE-4703:
------------------------------
I will apply this patch this afternoon unless objection.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139774#comment-13139774 ]
nkeywal commented on HBASE-4703:
--------------------------------
I don't have a generic solution. In some cases, it's possible to replace something like this:
{noformat}
for (i<1; i<nbTries; ++i){
if (cond())
break;
Thread.sleep(1s);
}
{noformat}
by
{noformat}
while (!timeout(5s) && !cond()){
Thread.sleep(0.1s);
}
{noformat}
On average the second solution takes 0.45s less than the first. Basically, in the tests we should be more aggressive with a 3 nodes cluster than in production with a 100 nodes one. I.e.: it makes sense to test a condition 10 times per second on the tests and 1 time per second in prod. Ideally, there should be no sleep at all, just a wait for the right event.
In tha patch, in most cases, the modifications are of the first type; just increasing the number of tests per second, but not changing the timeout.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4703:
-------------------------
Status: Patch Available (was: Open)
Submitting test to run it against patch build
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139817#comment-13139817 ]
Hadoop QA commented on HBASE-4703:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501528/20111030_4703_all.v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 186 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/105//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/105//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/105//console
This message is automatically generated.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141783#comment-13141783 ]
stack commented on HBASE-4703:
------------------------------
TestAdmin passes for me. I need to run it a few times?
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Attachment: 20111030_4703_all.patch
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Open (was: Patch Available)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139816#comment-13139816 ]
Ted Yu commented on HBASE-4703:
-------------------------------
I got the following test failures for the attached patch.
These two failed both in test suite and individually.
{code}
testFlushCommitsWithAbort(org.apache.hadoop.hbase.client.TestMultiParallel) Time elapsed: 45.25 sec <<< ERROR!
java.io.IOException: HRegionInfo was null or empty in .META., row=keyvalues={multi_test_table,ddd,1320006349447.09bf3a7042e4b84494f49b21d8e6f771./info:server/1320006349957/Put/vlen=19, multi_test_table,ddd,1320006349447.09bf3a7042e4b84494f49b21d8e6f771./info:serverstartcode/1320006349957/Put/vlen=8}
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:908)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:769)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1429)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1314)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:878)
at org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:240)
at org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:207)
{code}
{code}
testThreeRSAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting) Time elapsed: 91.605 sec <<< FAILURE!
java.lang.AssertionError:
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testThreeRSAbort(TestDistributedLogSplitting.java:139)
{code}
The following failed in test suite but succeeded individually:
{code}
testMasterFailoverWithMockedRITOnDeadRS(org.apache.hadoop.hbase.master.TestMasterFailover) Time elapsed: 180.011 sec <<< ERROR!
java.lang.Exception: test timed out after 180000 milliseconds
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.MiniHBaseCluster.waitForActiveAndReadyMaster(MiniHBaseCluster.java:385)
at org.apache.hadoop.hbase.master.TestMasterFailover.testMasterFailoverWithMockedRITOnDeadRS(TestMasterFailover.java:923)
{code}
{code}
testRowRange(org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol) Time elapsed: 0.037 sec <<< FAILURE!
java.lang.AssertionError: Results should contain region test,ccc,1320009243019.ffa363645a679c2c4caf69ede87bdfe3. for row 'ccc'
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol.verifyRegionResults(TestServerCustomProtocol.java:335)
at org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol.verifyRegionResults(TestServerCustomProtocol.java:327)
at org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol.testRowRange(TestServerCustomProtocol.java:213)
{code}
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141219#comment-13141219 ]
Ted Yu commented on HBASE-4703:
-------------------------------
I started new build for TRUNK where TestHFileOutputFormat passed but TestAdmin still hung.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4703:
-------------------------
Fix Version/s: 0.92.0
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139747#comment-13139747 ]
stack commented on HBASE-4703:
------------------------------
I'm not sure there is much to discuss about the recommendations made above by N; they seem commonsensical. N, we should make sure your patterns make it out into documentation -- there is a dev section in the manual that we should add them to.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Open (was: Patch Available)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Description:
Global:
- when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
- 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
- Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
Local issues:
- TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
- TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
- TestMemoryBoundedLogMessageBuffer useless System.out.println
- io.hfile.TestReseekTo useless System.out.println
- TestTableInputFormat does not shutdown the cluster
- testGlobalMemStore does not shutdown the cluster
- rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
- HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
- TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
was:
Global:
- when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
- 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
- tmp dir management: to allow multiple tests using dir to run on the same machine without conflicting, the tmp dir is set once per JVM. This should fix HBASE-3672 as well, because the map reduce cluster is configured to use the hadoop tmp dir
- Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
Local issues:
- TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
- TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
- TestMemoryBoundedLogMessageBuffer useless System.out.println
- io.hfile.TestReseekTo useless System.out.println
- TestTableInputFormat does not shutdown the cluster
- testGlobalMemStore does not shutdown the cluster
- rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
- HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
- TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Attachment: 20111030_4703_all.v4.patch
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141437#comment-13141437 ]
nkeywal commented on HBASE-4703:
--------------------------------
I will have a look.
Le 1 nov. 2011 16:01, "Ted Yu (Commented) (JIRA)" <ji...@apache.org> a
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141050#comment-13141050 ]
Hudson commented on HBASE-4703:
-------------------------------
Integrated in HBase-TRUNK #2394 (See [https://builds.apache.org/job/HBase-TRUNK/2394/])
HBASE-4703 Improvements in tests
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/pom.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TimestampTestBase.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultipleTimestamps.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestScannerTimeout.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestTimestampsFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestAggregateProtocol.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpoint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestColumnRangeFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TsvImporterCustomTestMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/monitoring/TestMemoryBoundedLogMessageBuffer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreLAB.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSeekOptimizations.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestGzipFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestScannersWithFilters.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestSchemaResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestTransform.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/TestVersionResource.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteAdmin.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestIdLock.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitter.java
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Patch Available (was: Open)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Attachment: 20111030_4703_all.v2.patch
taking into account ted's comment (with --no-prefix).
Ok for your other comments, I will write a general mail.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4703:
-------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed branch and trunk. Thanks for the patch nkeyway. Want to write a few notes in another issue and we can stick your pattern going forward writing tests into the manual. Good stuff.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141063#comment-13141063 ]
nkeywal commented on HBASE-4703:
--------------------------------
There are two issues in the central build:
- TestAdmin didn't finish
- TestHFileOutputFormat failed.
These two tests ran ok 3 times in the tests (+ multiple time on my env).
I can remove/revert my changes in TestHFileOutputFormat of course...
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140335#comment-13140335 ]
nkeywal commented on HBASE-4703:
--------------------------------
The two errors in the report above are caused by a "java.io.IOException: Too many open files" (even if it's hidden in the logs for testWorkerAbort). I tend to think that this patch is now ok.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139763#comment-13139763 ]
Ted Yu commented on HBASE-4703:
-------------------------------
bq. 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility
I am not sure if we can remove sleep based synchronisation.
As the analysis in HBASE-4690 showed, the timing of operations in cluster for different hardware / OS combinations would be different.
I guess we don't want to make test code too complicated so that test code can be perfectly synchronous with minicluster.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141961#comment-13141961 ]
nkeywal commented on HBASE-4703:
--------------------------------
Created HBASE-4724 to track this.
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141024#comment-13141024 ]
nkeywal commented on HBASE-4703:
--------------------------------
For the documentation, I created HBASE-4712
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Patch Available (was: Open)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Open (was: Patch Available)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4703) Improvements in tests
Posted by "nkeywal (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-4703:
---------------------------
Status: Open (was: Patch Available)
> Improvements in tests
> ---------------------
>
> Key: HBASE-4703
> URL: https://issues.apache.org/jira/browse/HBASE-4703
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.92.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch, 20111030_4703_all.v3.patch, 20111030_4703_all.v4.patch
>
>
> Global:
> - when possible, make the test using the default cluster configuration for the number of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared cluster configuration.
> - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility, TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache, TestServerCustomProtocol, TestReplicationSink)
> - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop. Not done for tests that rely on the WAL.
>
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira