You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/06/14 23:37:52 UTC

ZooKeeper-trunk - Build # 3428 - Still Failing

See https://builds.apache.org/job/ZooKeeper-trunk/3428/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 63.43 MB...]
    [junit] 2017-06-14 23:37:27,577 [myid:127.0.0.1:19427] - INFO  [main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:19427. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-06-14 23:37:27,578 [myid:] - INFO  [New I/O boss #10164:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19427
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-06-14 23:37:27,579 [myid:] - WARN  [New I/O boss #10164:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 0xdfa90995] EXCEPTION: java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19427
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19427
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-06-14 23:37:27,579 [myid:] - INFO  [New I/O boss #10164:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2017-06-14 23:37:27,579 [myid:127.0.0.1:19427] - INFO  [main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1231] - channel for sessionid 0x20275a2fdc60000 is lost, closing socket connection and attempting reconnect
    [junit] Running org.apache.zookeeper.test.NettyNettySuiteTest in thread 4
    [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec, Thread: 4, Class: org.apache.zookeeper.test.NettyNettySuiteTest
    [junit] Test org.apache.zookeeper.test.NettyNettySuiteTest FAILED (timeout)

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1338: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1223: Tests failed!

Total time: 20 minutes 33 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
2 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.ReconfigDuringLeaderSyncTest.testDuringLeaderSync

Error Message:
zoo.cfg.dynamic.next is not deleted.

Stack Trace:
junit.framework.AssertionFailedError: zoo.cfg.dynamic.next is not deleted.
	at org.apache.zookeeper.server.quorum.ReconfigDuringLeaderSyncTest.testDuringLeaderSync(ReconfigDuringLeaderSyncTest.java:165)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.NettyNettySuiteTest.testRoleChange

Error Message:
Timeout occurred. Please note the time in the report does not reflect the time until the timeout.

Stack Trace:
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
	at java.lang.Thread.run(Thread.java:745)

ZooKeeper-trunk - Build # 3430 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3430/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 80.12 MB...]
    [junit] 	at java.lang.Object.wait(Native Method)
    [junit] 	at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:563)
    [junit] 	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1127)
    [junit] 2017-06-15 23:26:59,723 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):Leader@626] - Shutting down
    [junit] 2017-06-15 23:26:59,723 [myid:] - WARN  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):QuorumPeer@1161] - PeerState set to LOOKING
    [junit] 2017-06-15 23:26:59,723 [myid:] - WARN  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):QuorumPeer@1143] - QuorumPeer main thread exited
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5]
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.5]
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.1]
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.2]
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.3]
    [junit] 2017-06-15 23:26:59,724 [myid:] - INFO  [/127.0.0.1:24850:QuorumCnxManager$Listener@667] - Leaving listener
    [junit] 2017-06-15 23:26:59,725 [myid:] - INFO  [main:QuorumUtil@254] - Shutting down leader election QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled)
    [junit] 2017-06-15 23:26:59,725 [myid:] - INFO  [main:QuorumUtil@259] - Waiting for QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled) to exit thread
    [junit] 2017-06-15 23:26:59,725 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24836
    [junit] 2017-06-15 23:26:59,726 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24836 is no longer accepting client connections
    [junit] 2017-06-15 23:26:59,726 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24839
    [junit] 2017-06-15 23:26:59,726 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24839 is no longer accepting client connections
    [junit] 2017-06-15 23:26:59,726 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24842
    [junit] 2017-06-15 23:26:59,726 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24842 is no longer accepting client connections
    [junit] 2017-06-15 23:26:59,727 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24845
    [junit] 2017-06-15 23:26:59,727 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24845 is no longer accepting client connections
    [junit] 2017-06-15 23:26:59,727 [myid:] - INFO  [main:FourLetterWordMain@85] - connecting to 127.0.0.1 24848
    [junit] 2017-06-15 23:26:59,727 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24848 is no longer accepting client connections
    [junit] 2017-06-15 23:26:59,729 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testRemoveOneAsynchronous
    [junit] 2017-06-15 23:26:59,729 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testRemoveOneAsynchronous
    [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 321.443 sec, Thread: 6, Class: org.apache.zookeeper.test.ReconfigTest
    [junit] 2017-06-15 23:26:59,766 [myid:127.0.0.1:24738] - INFO  [main-SendThread(127.0.0.1:24738):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24738. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-06-15 23:26:59,767 [myid:127.0.0.1:24738] - WARN  [main-SendThread(127.0.0.1:24738):ClientCnxn$SendThread@1235] - Session 0x204d7c26ed50000 for server 127.0.0.1/127.0.0.1:24738, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk@2/build.xml:1338: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk@2/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk@2/build.xml:1223: Tests failed!

Total time: 10 minutes 35 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
67 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentObserverIsParticipantInNewConfig

Error Message:
client could not connect to reestablished quorum: giving up after 30+ seconds.

Stack Trace:
junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
	at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:183)
	at org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentObserverIsParticipantInNewConfig(ReconfigRecoveryTest.java:487)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest

Error Message:
Server 4 is not up

Stack Trace:
junit.framework.AssertionFailedError: Server 4 is not up
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startServer(StandaloneDisabledTest.java:175)
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startObservers(StandaloneDisabledTest.java:222)
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest(StandaloneDisabledTest.java:111)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startObserver

Error Message:
Observer was able to start by itself!

Stack Trace:
junit.framework.AssertionFailedError: Observer was able to start by itself!
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startObserver(StandaloneDisabledTest.java:260)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncByDiffAfterFollowerCrashes

Error Message:
Waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: Waiting for server up
	at org.apache.zookeeper.test.QuorumUtil.startThenShutdown(QuorumUtil.java:231)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncByDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:490)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testFollowerSendsLastZxid

Error Message:
Failed to connect to ZooKeeper server.

Stack Trace:
java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper server.
	at org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:149)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.createTestableClient(FollowerResyncConcurrencyTest.java:609)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.createTestableClient(FollowerResyncConcurrencyTest.java:599)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testFollowerSendsLastZxid(FollowerResyncConcurrencyTest.java:680)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.MultiTransactionTest.testSetData[1]

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:761)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ClientBase.startServerInstance(ClientBase.java:421)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:527)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.MultiTransactionTest.setUp(MultiTransactionTest.java:81)


FAILED:  org.apache.zookeeper.test.MultiTransactionTest.testCreateDelete[1]

Error Message:
KeeperErrorCode = ConnectionLoss

Stack Trace:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.test.MultiTransactionTest.multi(MultiTransactionTest.java:113)
	at org.apache.zookeeper.test.MultiTransactionTest.testCreateDelete(MultiTransactionTest.java:471)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testClientCleanup

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError
	at org.apache.zookeeper.test.ClientTest.testClientCleanup(ClientTest.java:807)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testClientwithoutWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testSequentialNodeData

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testSequentialNodeNames

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testClientWithWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testTestability

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testNullAuthId

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testNonExistingOpCode

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testDeleteWithChildren

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:151)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne(ReconfigTest.java:252)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:184)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.AsyncHammerTest.setUp(AsyncHammerTest.java:52)
	at org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer(AsyncHammerTest.java:202)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncSetACLFailure_BadVersion

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.AsyncOpsTest.testAsyncDelete

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
	at org.apache.zookeeper.test.AsyncOpsTest.setUp(AsyncOpsTest.java:48)


FAILED:  org.apache.zookeeper.test.ClientTest.testClientCleanup

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError
	at org.apache.zookeeper.test.ClientTest.testClientCleanup(ClientTest.java:807)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ClientTest.testClientwithoutWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testSequentialNodeData

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testSequentialNodeNames

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testClientWithWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testTestability

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testNullAuthId

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testNonExistingOpCode

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testDeleteWithChildren

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ClientTest.testPathValidation

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ClientTest.testLargeNodeData

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)


FAILED:  org.apache.zookeeper.test.ClientTest.testMutipleWatcherObjs

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testFourLetterWords

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testSetTraceMask

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithLocal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.ZooKeeperServer.setupRequestProcessors(ZooKeeperServer.java:467)
	at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:453)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:511)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ClientBase.startServerInstance(ClientBase.java:421)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:527)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoReset(WatcherTest.java:296)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal(WatcherTest.java:254)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithLocal(WatcherTest.java:270)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ObserverTest.testObserver

Error Message:
waiting for server 2 being up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server 2 being up
	at org.apache.zookeeper.test.ObserverTest.testObserver(ObserverTest.java:147)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumMajorityTest.unknown

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.

Stack Trace:
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.QuorumQuotaTest.unknown

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.

Stack Trace:
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.QuorumZxidSyncTest.testBehindLeader

Error Message:
Servers didn't come up

Stack Trace:
junit.framework.AssertionFailedError: Servers didn't come up
	at org.apache.zookeeper.test.QuorumZxidSyncTest.testBehindLeader(QuorumZxidSyncTest.java:88)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
client could not connect to reestablished quorum: giving up after 30+ seconds.

Stack Trace:
junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
	at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:183)
	at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:841)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo

Error Message:
client could not connect to reestablished quorum: giving up after 30+ seconds.

Stack Trace:
junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
	at org.apache.zookeeper.test.ReconfigTest.reconfig(ReconfigTest.java:93)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:384)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateConsOutput

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateSocketTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateStatOutput

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.NullDataTest.testNullData

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
	at org.apache.zookeeper.test.NullDataTest.setUp(NullDataTest.java:43)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:816)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:589)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader(ReconfigTest.java:718)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion(ReconfigTest.java:893)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne(ReconfigTest.java:252)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:332)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower(ReconfigTest.java:714)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
	at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne(ReconfigTest.java:914)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRoleChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.quorum.QuorumPeer.createElectionAlgorithm(QuorumPeer.java:952)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection(QuorumPeer.java:843)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:779)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:211)
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
	at org.apache.zookeeper.test.ReconfigTest.testRoleChange(ReconfigTest.java:514)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testBulkReconfig

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testBulkReconfig(ReconfigTest.java:432)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange(ReconfigTest.java:991)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:471)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testMinMaxSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionMove

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeoutAccess

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSession

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionStateNoDupStateReporting

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.ZooKeeperServer.setupRequestProcessors(ZooKeeperServer.java:465)
	at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:453)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:511)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:78)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.<init>(NioServerBoss.java:49)
	at org.jboss.netty.channel.socket.nio.NioServerBossPool.newBoss(NioServerBossPool.java:55)
	at org.jboss.netty.channel.socket.nio.NioServerBossPool.newBoss(NioServerBossPool.java:26)
	at org.jboss.netty.channel.socket.nio.AbstractNioBossPool.init(AbstractNioBossPool.java:75)
	at org.jboss.netty.channel.socket.nio.NioServerBossPool.<init>(NioServerBossPool.java:40)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:179)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoReset(WatcherTest.java:357)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal(WatcherTest.java:254)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:338)
	at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
	at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
	at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
	at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:310)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:273)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:118)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:95)
	at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:403)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:54)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumHammerTest.setUp(QuorumHammerTest.java:36)


FAILED:  org.apache.zookeeper.test.QuorumTest.testACLs

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)

ZooKeeper-trunk - Build # 3429 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3429/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 92.32 MB...]
    [junit] 2017-06-15 23:26:03,006 [myid:127.0.0.1:11312] - INFO  [main-SendThread(127.0.0.1:11312):ClientCnxn$SendThread@1231] - channel for sessionid 0x304d7bec4620001 is lost, closing socket connection and attempting reconnect
    [junit] 2017-06-15 23:26:03,191 [myid:127.0.0.1:11318] - INFO  [main-SendThread(127.0.0.1:11318):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11318. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-06-15 23:26:03,192 [myid:] - INFO  [New I/O boss #12642:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:11318
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-06-15 23:26:03,192 [myid:] - WARN  [New I/O boss #12642:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 0x7685143f] EXCEPTION: java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:11318
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:11318
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-06-15 23:26:03,192 [myid:] - INFO  [New I/O boss #12642:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2017-06-15 23:26:03,192 [myid:127.0.0.1:11318] - INFO  [main-SendThread(127.0.0.1:11318):ClientCnxn$SendThread@1231] - channel for sessionid 0x504d7bec4620000 is lost, closing socket connection and attempting reconnect
    [junit] Test org.apache.zookeeper.test.NettyNettySuiteTest FAILED

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1338: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1219: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1223: Tests failed!

Total time: 9 minutes 48 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2786
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
57 tests failed.
FAILED:  org.apache.zookeeper.server.ZxidRolloverTest.testRolloverThenFollowerRestart

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
	at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
	at org.apache.zookeeper.server.ZxidRolloverTest.start(ZxidRolloverTest.java:157)
	at org.apache.zookeeper.server.ZxidRolloverTest.testRolloverThenFollowerRestart(ZxidRolloverTest.java:345)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.server.admin.JettyAdminServerTest.testQuorum

Error Message:
Connection refused

Stack Trace:
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at java.net.Socket.connect(Socket.java:528)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
	at sun.net.www.http.HttpClient.New(HttpClient.java:308)
	at sun.net.www.http.HttpClient.New(HttpClient.java:326)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:933)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1301)
	at java.net.URL.openStream(URL.java:1041)
	at org.apache.zookeeper.server.admin.JettyAdminServerTest.queryAdminServer(JettyAdminServerTest.java:158)
	at org.apache.zookeeper.server.admin.JettyAdminServerTest.queryAdminServer(JettyAdminServerTest.java:150)
	at org.apache.zookeeper.server.admin.JettyAdminServerTest.testQuorum(JettyAdminServerTest.java:132)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError
	at org.apache.zookeeper.test.ReconfigTest.testServerHasConfig(ReconfigTest.java:141)
	at org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig(ReconfigRecoveryTest.java:220)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testNextConfigCompletion

Error Message:
waiting for server 0 being up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server 0 being up
	at org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testNextConfigCompletion(ReconfigRecoveryTest.java:81)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncTest.testAsync

Error Message:
waiting for server down

Stack Trace:
junit.framework.AssertionFailedError: waiting for server down
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:373)
	at org.apache.zookeeper.test.AsyncTest.tearDown(AsyncTest.java:56)


FAILED:  org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:780)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:211)
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.followerResyncCrashTest(FollowerResyncConcurrencyTest.java:200)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:176)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testFollowerWatcherResync

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testFollowerWatcherResync(FollowerResyncConcurrencyTest.java:713)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
client could not connect to reestablished quorum: giving up after 30+ seconds.

Stack Trace:
junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
	at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:183)
	at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:841)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at java.net.PlainSocketImpl.socketBind(Native Method)
	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
	at java.net.ServerSocket.bind(ServerSocket.java:376)
	at java.net.ServerSocket.bind(ServerSocket.java:330)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:779)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader(ReconfigTest.java:718)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo

Error Message:
Failed to bind to: /127.0.0.1:11391

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: /127.0.0.1:11391
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:211)
	at org.apache.zookeeper.test.QuorumUtil.restart(QuorumUtil.java:215)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:422)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ChrootClientTest.testNullAuthId

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.ZooKeeperServer.setupRequestProcessors(ZooKeeperServer.java:467)
	at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:453)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:511)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ClientBase.startServerInstance(ClientBase.java:421)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:527)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.ChrootClientTest.setUp(ChrootClientTest.java:37)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateConsOutput

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateSocketTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.FourLetterWordsTest.testValidateStatOutput

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)


FAILED:  org.apache.zookeeper.test.NullDataTest.testNullData

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
	at org.apache.zookeeper.test.NullDataTest.setUp(NullDataTest.java:43)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:816)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:589)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader(ReconfigTest.java:718)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion(ReconfigTest.java:893)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne(ReconfigTest.java:250)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:332)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
	at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower(ReconfigTest.java:714)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne(ReconfigTest.java:912)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRoleChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRoleChange(ReconfigTest.java:512)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testBulkReconfig

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testBulkReconfig(ReconfigTest.java:432)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange(ReconfigTest.java:991)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:471)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testMinMaxSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionMove

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeoutAccess

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSession

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionStateNoDupStateReporting

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherCorrectness

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithGlobal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoReset(WatcherTest.java:387)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithGlobal(WatcherTest.java:245)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithGlobal(WatcherTest.java:264)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:383)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:147)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoReset(WatcherTest.java:357)
	at org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal(WatcherTest.java:254)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ObserverLETest.testLEWithObserver

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.ObserverLETest.establishThreeParticipantOneObserverEnsemble(ObserverLETest.java:37)


FAILED:  org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumHammerTest.setUp(QuorumHammerTest.java:36)


FAILED:  org.apache.zookeeper.test.QuorumMajorityTest.testMajQuorums

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)


FAILED:  org.apache.zookeeper.test.QuorumQuotaTest.testQuotaWithQuorum

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:338)
	at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
	at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
	at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
	at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:310)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:273)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:118)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:95)
	at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:403)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:54)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)


FAILED:  org.apache.zookeeper.test.QuorumQuotaTest.testNull

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at sun.misc.GC$Daemon$1.run(GC.java:140)
	at sun.misc.GC$Daemon$1.run(GC.java:131)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.GC$Daemon.create(GC.java:144)
	at sun.misc.GC.setLatencyTarget(GC.java:157)
	at sun.misc.GC.access$500(GC.java:41)
	at sun.misc.GC$LatencyRequest.adjustLatencyIfNeeded(GC.java:193)
	at sun.misc.GC$LatencyRequest.<init>(GC.java:218)
	at sun.misc.GC$LatencyRequest.<init>(GC.java:173)
	at sun.misc.GC.requestLatency(GC.java:271)
	at sun.rmi.transport.ObjectTable.incrementKeepAliveCount(ObjectTable.java:292)
	at sun.rmi.transport.ObjectTable.putTarget(ObjectTable.java:196)
	at sun.rmi.transport.Transport.exportObject(Transport.java:105)
	at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:259)
	at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
	at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
	at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:310)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:273)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:118)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:95)
	at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:403)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:54)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)


FAILED:  org.apache.zookeeper.test.QuorumTest.testLeaderShutdown

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:780)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:210)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testACLs

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.QuorumTest.testPing

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at sun.rmi.transport.ObjectTable.incrementKeepAliveCount(ObjectTable.java:281)
	at sun.rmi.transport.ObjectTable.putTarget(ObjectTable.java:196)
	at sun.rmi.transport.Transport.exportObject(Transport.java:105)
	at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:259)
	at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
	at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
	at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:310)
	at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:273)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:118)
	at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:95)
	at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:403)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:54)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testNoLogBeforeLeaderEstablishment

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.QuorumTest.testClientwithoutWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:751)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:214)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testGetView

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:754)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:210)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testSequentialNodeNames

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:210)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testViewContains

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:210)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testSessionMoved

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:754)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1601)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:214)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testClientWithWatcherObj

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger.start(FastLeaderElection.java:520)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.start(FastLeaderElection.java:637)
	at org.apache.zookeeper.server.quorum.QuorumPeer.createElectionAlgorithm(QuorumPeer.java:954)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection(QuorumPeer.java:843)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:779)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:210)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testMultiToFollower

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
	at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumUtilTest.validateAllMXBeanAreUnregistered

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:151)
	at org.apache.zookeeper.test.QuorumUtilTest.validateAllMXBeanAreUnregistered(QuorumUtilTest.java:53)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReadOnlyModeTest.testSeekForRwServer

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
	at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
	at org.apache.zookeeper.test.ReadOnlyModeTest.testSeekForRwServer(ReadOnlyModeTest.java:280)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigExceptionTest.testReconfigEnabledWithSuperUser

Error Message:
Waiting for server down

Stack Trace:
junit.framework.AssertionFailedError: Waiting for server down
	at org.apache.zookeeper.test.QuorumUtil.shutdownAll(QuorumUtil.java:241)
	at org.apache.zookeeper.test.QuorumUtil.tearDown(QuorumUtil.java:306)
	at org.apache.zookeeper.test.ReconfigExceptionTest.tearDown(ReconfigExceptionTest.java:80)


FAILED:  org.apache.zookeeper.test.WatchEventWhenAutoResetTest.testNodeDataChanged

Error Message:
expected:<NodeDataChanged> but was:<NodeDeleted>

Stack Trace:
junit.framework.AssertionFailedError: expected:<NodeDataChanged> but was:<NodeDeleted>
	at org.apache.zookeeper.test.WatchEventWhenAutoResetTest$EventsWatcher.assertEvent(WatchEventWhenAutoResetTest.java:67)
	at org.apache.zookeeper.test.WatchEventWhenAutoResetTest.testNodeDataChanged(WatchEventWhenAutoResetTest.java:126)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)