You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/05/09 20:02:28 UTC

ZooKeeper-trunk-openjdk7 - Build # 1463 - Still Failing

See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1463/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 61.58 MB...]
    [junit] 2017-05-09 20:02:15,862 [myid:127.0.0.1:22123] - INFO  [main-SendThread(127.0.0.1:22123):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:22123. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-09 20:02:15,862 [myid:127.0.0.1:22123] - WARN  [main-SendThread(127.0.0.1:22123):ClientCnxn$SendThread@1235] - Session 0x30419fd71390000 for server 127.0.0.1/127.0.0.1:22123, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-09 20:02:16,080 [myid:127.0.0.1:22117] - INFO  [main-SendThread(127.0.0.1:22117):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:22117. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-09 20:02:16,081 [myid:127.0.0.1:22117] - WARN  [main-SendThread(127.0.0.1:22117):ClientCnxn$SendThread@1235] - Session 0x10419fd71350000 for server 127.0.0.1/127.0.0.1:22117, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-09 20:02:16,590 [myid:] - INFO  [ProcessThread(sid:0 cport:22240)::PrepRequestProcessor@617] - Processed session termination for sessionid: 0x1041a00985b0000
    [junit] 2017-05-09 20:02:16,591 [myid:] - INFO  [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port22240,name1=Connections,name2=127.0.0.1,name3=0x1041a00985b0000]
    [junit] 2017-05-09 20:02:16,591 [myid:] - INFO  [main:ZooKeeper@1329] - Session: 0x1041a00985b0000 closed
    [junit] 2017-05-09 20:02:16,591 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x1041a00985b0000
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 197274
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2427
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:ClientBase@582] - tearDown starting
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:ClientBase@552] - STOPPING server
    [junit] 2017-05-09 20:02:16,592 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:22240
    [junit] 2017-05-09 20:02:16,597 [myid:] - INFO  [main:ZooKeeperServer@542] - shutting down
    [junit] 2017-05-09 20:02:16,597 [myid:] - ERROR [main:ZooKeeperServer@506] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-05-09 20:02:16,597 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-05-09 20:02:16,597 [myid:] - INFO  [main:PrepRequestProcessor@1008] - Shutting down
    [junit] 2017-05-09 20:02:16,597 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-05-09 20:02:16,597 [myid:] - INFO  [ProcessThread(sid:0 cport:22240)::PrepRequestProcessor@157] - PrepRequestProcessor exited loop!
    [junit] 2017-05-09 20:02:16,598 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-05-09 20:02:16,598 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-05-09 20:02:16,598 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port22240,name1=InMemoryDataTree]
    [junit] 2017-05-09 20:02:16,598 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port22240]
    [junit] 2017-05-09 20:02:16,598 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 22240
    [junit] 2017-05-09 20:02:16,599 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-05-09 20:02:16,604 [myid:127.0.0.1:21994] - INFO  [main-SendThread(127.0.0.1:21994):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:21994. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-09 20:02:16,605 [myid:127.0.0.1:21994] - WARN  [main-SendThread(127.0.0.1:21994):ClientCnxn$SendThread@1235] - Session 0x10419f9b0fe0000 for server 127.0.0.1/127.0.0.1:21994, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-09 20:02:16,607 [myid:] - INFO  [main:ClientBase@607] - fdcount after test is: 7148 at start it was 7148
    [junit] 2017-05-09 20:02:16,607 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-05-09 20:02:16,608 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 458.771 sec, Thread: 5, Class: org.apache.zookeeper.test.NioNettySuiteTest

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1223: Tests failed!

Total time: 10 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) ##############################
37 tests failed.
FAILED:  org.apache.zookeeper.test.LocalSessionsOnlyTest.testLocalSessionsOnFollower

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.LocalSessionsOnlyTest.setUp(LocalSessionsOnlyTest.java:55)


FAILED:  org.apache.zookeeper.test.NIOConnectionFactoryFdLeakTest.testFileDescriptorLeak

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.test.NIOConnectionFactoryFdLeakTest.testFileDescriptorLeak(NIOConnectionFactoryFdLeakTest.java:57)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncMulti

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncMultiSequential_NoSideEffect

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildrenEmpty

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetACLFailure_BadVersion

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreate

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncDelete

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildren2Failure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncExists

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetDataFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetACL

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreate2Failure_NoNode

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreateFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetACL

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncMultiFailure_NoSideEffect

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetData

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSync

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildren2Two

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreateFailure_NoChildForEphemeral

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncDeleteFailure_BadVersion

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncDeleteFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetDataFailure_BadVersion

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildrenFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreate2

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncExistsFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildren2Empty

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreateThree

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncMultiFailure_AllErrorResult

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:379)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:427)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:506)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:499)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncGetChildrenTwo

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreateFailure_NodeExists

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncCreate2Failure_NoChildForEphemeral

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetData

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetDataFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetACLFailure_NoNode

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:493)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testHammer

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.test.AsyncHammerTest.testHammer(AsyncHammerTest.java:176)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.AsyncHammerTest.setUp(AsyncHammerTest.java:52)
	at org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer(AsyncHammerTest.java:202)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

ZooKeeper-trunk-openjdk7 - Build # 1467 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1467/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 62.09 MB...]
    [junit] 2017-05-12 20:02:19,848 [myid:127.0.0.1:11348] - INFO  [main-SendThread(127.0.0.1:11348):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11348. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-12 20:02:19,849 [myid:127.0.0.1:11348] - WARN  [main-SendThread(127.0.0.1:11348):ClientCnxn$SendThread@1235] - Session 0x2042934d9c60000 for server 127.0.0.1/127.0.0.1:11348, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-12 20:02:19,890 [myid:127.0.0.1:11222] - INFO  [main-SendThread(127.0.0.1:11222):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11222. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-12 20:02:19,890 [myid:127.0.0.1:11222] - WARN  [main-SendThread(127.0.0.1:11222):ClientCnxn$SendThread@1235] - Session 0x1042931160c0000 for server 127.0.0.1/127.0.0.1:11222, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-12 20:02:19,933 [myid:127.0.0.1:11345] - INFO  [main-SendThread(127.0.0.1:11345):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11345. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-12 20:02:19,933 [myid:127.0.0.1:11345] - WARN  [main-SendThread(127.0.0.1:11345):ClientCnxn$SendThread@1235] - Session 0x1042934d9c60000 for server 127.0.0.1/127.0.0.1:11345, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-12 20:02:20,431 [myid:] - INFO  [ProcessThread(sid:0 cport:11468)::PrepRequestProcessor@617] - Processed session termination for sessionid: 0x104293837180000
    [junit] 2017-05-12 20:02:20,432 [myid:] - INFO  [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468,name1=Connections,name2=127.0.0.1,name3=0x104293837180000]
    [junit] 2017-05-12 20:02:20,432 [myid:] - INFO  [main:ZooKeeper@1329] - Session: 0x104293837180000 closed
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 214994
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2428
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x104293837180000
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main:ClientBase@582] - tearDown starting
    [junit] 2017-05-12 20:02:20,433 [myid:] - INFO  [main:ClientBase@552] - STOPPING server
    [junit] 2017-05-12 20:02:20,434 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:11468
    [junit] 2017-05-12 20:02:20,437 [myid:] - INFO  [main:ZooKeeperServer@542] - shutting down
    [junit] 2017-05-12 20:02:20,437 [myid:] - ERROR [main:ZooKeeperServer@506] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-05-12 20:02:20,437 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-05-12 20:02:20,437 [myid:] - INFO  [main:PrepRequestProcessor@1008] - Shutting down
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [ProcessThread(sid:0 cport:11468)::PrepRequestProcessor@157] - PrepRequestProcessor exited loop!
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468,name1=InMemoryDataTree]
    [junit] 2017-05-12 20:02:20,438 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468]
    [junit] 2017-05-12 20:02:20,439 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 11468
    [junit] 2017-05-12 20:02:20,439 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-05-12 20:02:20,448 [myid:] - INFO  [main:ClientBase@607] - fdcount after test is: 7139 at start it was 7139
    [junit] 2017-05-12 20:02:20,448 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-05-12 20:02:20,448 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 474.176 sec, Thread: 1, Class: org.apache.zookeeper.test.NioNettySuiteTest

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1223: Tests failed!

Total time: 11 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) ##############################
1 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.ReconfigDuringLeaderSyncTest.testDuringLeaderSync

Error Message:
zoo.cfg.dynamic.next is not deleted.

Stack Trace:
junit.framework.AssertionFailedError: zoo.cfg.dynamic.next is not deleted.
	at org.apache.zookeeper.server.quorum.ReconfigDuringLeaderSyncTest.testDuringLeaderSync(ReconfigDuringLeaderSyncTest.java:165)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)

ZooKeeper-trunk-openjdk7 - Build # 1466 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1466/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 57.55 MB...]
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-05-11 20:10:33,204 [myid:] - INFO  [New I/O boss #98:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2017-05-11 20:10:33,204 [myid:127.0.0.1:13915] - INFO  [main-SendThread(127.0.0.1:13915):ClientCnxn$SendThread@1231] - channel for sessionid 0x1042411d77f0000 is lost, closing socket connection and attempting reconnect
    [junit] 2017-05-11 20:10:33,365 [myid:127.0.0.1:14041] - INFO  [main-SendThread(127.0.0.1:14041):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:14041. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-11 20:10:33,365 [myid:] - INFO  [New I/O boss #15092:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:14041
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-05-11 20:10:33,366 [myid:] - WARN  [New I/O boss #15092:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 0xbc82c4dc] EXCEPTION: java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:14041
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:14041
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-05-11 20:10:33,366 [myid:] - INFO  [New I/O boss #15092:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2017-05-11 20:10:33,366 [myid:127.0.0.1:14041] - INFO  [main-SendThread(127.0.0.1:14041):ClientCnxn$SendThread@1231] - channel for sessionid 0x2042415e3d80000 is lost, closing socket connection and attempting reconnect
    [junit] Running org.apache.zookeeper.server.quorum.Zab1_0Test in thread 1
    [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec, Thread: 1, Class: org.apache.zookeeper.server.quorum.Zab1_0Test
    [junit] Test org.apache.zookeeper.server.quorum.Zab1_0Test FAILED (timeout)

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1223: Tests failed!

Total time: 16 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) ##############################
2 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.Zab1_0Test.testNormalObserverRun

Error Message:
Timeout occurred. Please note the time in the report does not reflect the time until the timeout.

Stack Trace:
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.WatchEventWhenAutoResetTest.testNodeDataChanged

Error Message:
expected:<NodeDataChanged> but was:<NodeDeleted>

Stack Trace:
junit.framework.AssertionFailedError: expected:<NodeDataChanged> but was:<NodeDeleted>
	at org.apache.zookeeper.test.WatchEventWhenAutoResetTest$EventsWatcher.assertEvent(WatchEventWhenAutoResetTest.java:67)
	at org.apache.zookeeper.test.WatchEventWhenAutoResetTest.testNodeDataChanged(WatchEventWhenAutoResetTest.java:117)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)

ZooKeeper-trunk-openjdk7 - Build # 1465 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1465/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 60.34 MB...]
    [junit] 2017-05-11 16:32:26,221 [myid:] - INFO  [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port19547,name1=Connections,name2=127.0.0.1,name3=0x104234e57280000]
    [junit] 2017-05-11 16:32:26,221 [myid:] - INFO  [main:ZooKeeper@1329] - Session: 0x104234e57280000 closed
    [junit] 2017-05-11 16:32:26,221 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x104234e57280000
    [junit] 2017-05-11 16:32:26,221 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 216040
    [junit] 2017-05-11 16:32:26,221 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2455
    [junit] 2017-05-11 16:32:26,222 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-05-11 16:32:26,222 [myid:] - INFO  [main:ClientBase@582] - tearDown starting
    [junit] 2017-05-11 16:32:26,222 [myid:] - INFO  [main:ClientBase@552] - STOPPING server
    [junit] 2017-05-11 16:32:26,222 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:19547
    [junit] 2017-05-11 16:32:26,226 [myid:] - INFO  [main:ZooKeeperServer@542] - shutting down
    [junit] 2017-05-11 16:32:26,226 [myid:] - ERROR [main:ZooKeeperServer@506] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-05-11 16:32:26,226 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [main:PrepRequestProcessor@1008] - Shutting down
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [ProcessThread(sid:0 cport:19547)::PrepRequestProcessor@157] - PrepRequestProcessor exited loop!
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port19547,name1=InMemoryDataTree]
    [junit] 2017-05-11 16:32:26,227 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port19547]
    [junit] 2017-05-11 16:32:26,228 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 19547
    [junit] 2017-05-11 16:32:26,228 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-05-11 16:32:26,235 [myid:] - INFO  [main:ClientBase@607] - fdcount after test is: 7201 at start it was 7201
    [junit] 2017-05-11 16:32:26,235 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-05-11 16:32:26,235 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 103, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 512.378 sec, Thread: 4, Class: org.apache.zookeeper.test.NioNettySuiteTest
    [junit] 2017-05-11 16:32:26,469 [myid:127.0.0.1:19376] - INFO  [main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:19376. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-11 16:32:26,470 [myid:127.0.0.1:19376] - WARN  [main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1235] - Session 0x1042348badd0001 for server 127.0.0.1/127.0.0.1:19376, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-11 16:32:26,478 [myid:127.0.0.1:19382] - INFO  [main-SendThread(127.0.0.1:19382):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:19382. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-11 16:32:26,479 [myid:127.0.0.1:19382] - WARN  [main-SendThread(127.0.0.1:19382):ClientCnxn$SendThread@1235] - Session 0x3042348bb630001 for server 127.0.0.1/127.0.0.1:19382, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-11 16:32:26,714 [myid:127.0.0.1:19394] - INFO  [main-SendThread(127.0.0.1:19394):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:19394. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-11 16:32:26,715 [myid:127.0.0.1:19394] - WARN  [main-SendThread(127.0.0.1:19394):ClientCnxn$SendThread@1235] - Session 0x7042348b99c0000 for server 127.0.0.1/127.0.0.1:19394, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] Test org.apache.zookeeper.test.NioNettySuiteTest FAILED

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1223: Tests failed!

Total time: 13 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) ##############################
5 tests failed.
FAILED:  org.apache.zookeeper.server.ZooKeeperServerMainTest.testWithOnlyMinSessionTimeout

Error Message:
Failed to establish zkclient connection!

Stack Trace:
junit.framework.AssertionFailedError: Failed to establish zkclient connection!
	at org.apache.zookeeper.server.ZooKeeperServerMainTest.verifySessionTimeOut(ZooKeeperServerMainTest.java:420)
	at org.apache.zookeeper.server.ZooKeeperServerMainTest.testWithOnlyMinSessionTimeout(ZooKeeperServerMainTest.java:370)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.FourLetterWordsQuorumTest.testNull

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChange

Error Message:
Existing client disconnected when client port changed!

Stack Trace:
junit.framework.AssertionFailedError: Existing client disconnected when client port changed!
	at org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:621)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
Waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: Waiting for server up
	at org.apache.zookeeper.test.QuorumUtil.restart(QuorumUtil.java:216)
	at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:859)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumZxidSyncTest.testLateLogs

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.setUp(QuorumZxidSyncTest.java:40)

ZooKeeper-trunk-openjdk7 - Build # 1464 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1464/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 56.17 MB...]
    [junit] 2017-05-10 21:03:11,802 [myid:127.0.0.1:24687] - INFO  [main-SendThread(127.0.0.1:24687):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24687. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-10 21:03:11,803 [myid:127.0.0.1:24687] - WARN  [main-SendThread(127.0.0.1:24687):ClientCnxn$SendThread@1235] - Session 0x1041ef25c970000 for server 127.0.0.1/127.0.0.1:24687, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-10 21:03:11,913 [myid:127.0.0.1:24810] - INFO  [main-SendThread(127.0.0.1:24810):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24810. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-10 21:03:11,914 [myid:127.0.0.1:24810] - WARN  [main-SendThread(127.0.0.1:24810):ClientCnxn$SendThread@1235] - Session 0x1041ef6a7c50000 for server 127.0.0.1/127.0.0.1:24810, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-10 21:03:11,920 [myid:127.0.0.1:24816] - INFO  [main-SendThread(127.0.0.1:24816):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24816. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-05-10 21:03:11,921 [myid:127.0.0.1:24816] - WARN  [main-SendThread(127.0.0.1:24816):ClientCnxn$SendThread@1235] - Session 0x3041ef6a7cb0000 for server 127.0.0.1/127.0.0.1:24816, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-05-10 21:03:12,288 [myid:] - INFO  [ProcessThread(sid:0 cport:24933)::PrepRequestProcessor@617] - Processed session termination for sessionid: 0x1041ef9cef00000
    [junit] 2017-05-10 21:03:12,289 [myid:] - INFO  [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port24933,name1=Connections,name2=127.0.0.1,name3=0x1041ef9cef00000]
    [junit] 2017-05-10 21:03:12,290 [myid:] - INFO  [main:ZooKeeper@1329] - Session: 0x1041ef9cef00000 closed
    [junit] 2017-05-10 21:03:12,290 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x1041ef9cef00000
    [junit] 2017-05-10 21:03:12,290 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 204613
    [junit] 2017-05-10 21:03:12,290 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2427
    [junit] 2017-05-10 21:03:12,290 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-05-10 21:03:12,291 [myid:] - INFO  [main:ClientBase@582] - tearDown starting
    [junit] 2017-05-10 21:03:12,291 [myid:] - INFO  [main:ClientBase@552] - STOPPING server
    [junit] 2017-05-10 21:03:12,291 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:24933
    [junit] 2017-05-10 21:03:12,297 [myid:] - INFO  [main:ZooKeeperServer@542] - shutting down
    [junit] 2017-05-10 21:03:12,297 [myid:] - ERROR [main:ZooKeeperServer@506] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-05-10 21:03:12,297 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-05-10 21:03:12,297 [myid:] - INFO  [main:PrepRequestProcessor@1008] - Shutting down
    [junit] 2017-05-10 21:03:12,298 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-05-10 21:03:12,298 [myid:] - INFO  [ProcessThread(sid:0 cport:24933)::PrepRequestProcessor@157] - PrepRequestProcessor exited loop!
    [junit] 2017-05-10 21:03:12,298 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-05-10 21:03:12,298 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-05-10 21:03:12,298 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port24933,name1=InMemoryDataTree]
    [junit] 2017-05-10 21:03:12,299 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port24933]
    [junit] 2017-05-10 21:03:12,299 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24933
    [junit] 2017-05-10 21:03:12,299 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-05-10 21:03:12,315 [myid:] - INFO  [main:ClientBase@607] - fdcount after test is: 7137 at start it was 7137
    [junit] 2017-05-10 21:03:12,316 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-05-10 21:03:12,316 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 494.656 sec, Thread: 6, Class: org.apache.zookeeper.test.NioNettySuiteTest

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-openjdk7/build.xml:1223: Tests failed!

Total time: 11 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) ##############################
1 tests failed.
FAILED:  org.apache.zookeeper.test.LETest.testLE

Error Message:
Thread 5 got 27 expected 28

Stack Trace:
junit.framework.AssertionFailedError: Thread 5 got 27 expected 28
	at org.apache.zookeeper.test.LETest.testLE(LETest.java:135)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)