You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/11/14 23:26:42 UTC

ZooKeeper-trunk - Build # 3608 - Still Failing

See https://builds.apache.org/job/ZooKeeper-trunk/3608/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 65.77 MB...]
    [junit] 2017-11-14 23:26:31,164 [myid:] - INFO  [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30319,name1=Connections,name2=127.0.0.1,name3=0x107e68b3bd20000]
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:ZooKeeper@1332] - Session: 0x107e68b3bd20000 closed
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x107e68b3bd20000
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 195161
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2432
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:ClientBase@601] - tearDown starting
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:ClientBase@571] - STOPPING server
    [junit] 2017-11-14 23:26:31,165 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:30319
    [junit] 2017-11-14 23:26:31,171 [myid:] - INFO  [main:ZooKeeperServer@545] - shutting down
    [junit] 2017-11-14 23:26:31,171 [myid:] - ERROR [main:ZooKeeperServer@509] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-11-14 23:26:31,171 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [main:PrepRequestProcessor@1010] - Shutting down
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [ProcessThread(sid:0 cport:30319)::PrepRequestProcessor@155] - PrepRequestProcessor exited loop!
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-11-14 23:26:31,172 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30319,name1=InMemoryDataTree]
    [junit] 2017-11-14 23:26:31,173 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30319]
    [junit] 2017-11-14 23:26:31,173 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 30319
    [junit] 2017-11-14 23:26:31,173 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-11-14 23:26:31,183 [myid:] - INFO  [main:ClientBase@626] - fdcount after test is: 7153 at start it was 7149
    [junit] 2017-11-14 23:26:31,183 [myid:] - INFO  [main:ClientBase@628] - sleeping for 20 secs
    [junit] 2017-11-14 23:26:31,183 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-11-14 23:26:31,183 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 105, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 399.3 sec, Thread: 8, Class: org.apache.zookeeper.test.NioNettySuiteTest
    [junit] 2017-11-14 23:26:31,556 [myid:127.0.0.1:30122] - INFO  [main-SendThread(127.0.0.1:30122):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:30122. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-14 23:26:31,557 [myid:127.0.0.1:30122] - WARN  [main-SendThread(127.0.0.1:30122):ClientCnxn$SendThread@1235] - Session 0x107e6858e0c0001 for server 127.0.0.1/127.0.0.1:30122, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] Test org.apache.zookeeper.test.NioNettySuiteTest FAILED

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1267: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1271: Tests failed!

Total time: 9 minutes 47 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
3 tests failed.
FAILED:  org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncByDiffAfterFollowerCrashes

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1607)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumUtil.startThenShutdown(QuorumUtil.java:230)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncByDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:490)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ClientTest.testClientCleanup

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError
	at org.apache.zookeeper.test.ClientTest.testClientCleanup(ClientTest.java:807)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:746)
	at org.apache.zookeeper.server.quorum.QuorumPeer.startServerCnxnFactory(QuorumPeer.java:1607)
	at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:772)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:211)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:121)
	at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:79)
	at org.apache.zookeeper.test.QuorumHammerTest.setUp(QuorumHammerTest.java:36)

ZooKeeper-trunk - Build # 3618 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3618/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 64.99 MB...]
     [exec]  : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 11662 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1164 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1012 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1031 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1069 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1100 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17361 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1019 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3071 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 33339 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1099 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 19482 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 19626 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1251 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4519 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 5938 : OK
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4018
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] Makefile:1744: recipe for target 'check-TESTS' failed
     [exec] make[1]: Leaving directory '/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] Makefile:2000: recipe for target 'check-am' failed
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (31285) - No such process
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 15 minutes 20 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2806
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3617 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3617/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 65.55 MB...]
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 11722 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1156 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1107 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1031 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1123 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1104 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17368 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1035 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3079 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 38753 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1056 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 18530 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 18561 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1244 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4518 : OK
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 5818 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4017
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] Makefile:1744: recipe for target 'check-TESTS' failed
     [exec] make[1]: Leaving directory '/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] Makefile:2000: recipe for target 'check-am' failed
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (9367) - No such process
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 15 minutes 31 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2795
[JIRA] Updating issue ZOOKEEPER-2944
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3616 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3616/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 67.18 MB...]
     [exec] Log Message Received: [2017-11-20 23:31:58,952:18590(0x2abdb0290540):ZOO_INFO@log_env@1097: Client environment:user.home=/home/jenkins]
     [exec] Log Message Received: [2017-11-20 23:31:58,952:18590(0x2abdb0290540):ZOO_INFO@log_env@1109: Client environment:user.dir=/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit]
     [exec] Log Message Received: [2017-11-20 23:31:58,952:18590(0x2abdb0290540):ZOO_INFO@zookeeper_init_internal@1152: Initiating client connection, host=127.0.0.1:22181 sessionTimeout=10000 watcher=0x461d20 sessionId=0 sessionPasswd=<null> context=0x7ffed3ce82e0 flags=0]
     [exec] Log Message Received: [2017-11-20 23:31:58,953:18590(0x2abdb22f1700):ZOO_INFO@check_events@2425: initiated connection to server [127.0.0.1:22181]]
     [exec] Log Message Received: [2017-11-20 23:31:58,970:18590(0x2abdb22f1700):ZOO_INFO@check_events@2477: session establishment complete on server [127.0.0.1:22181], sessionId=0x100337f6256000f, negotiated timeout=10000 ]
     [exec]  : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 10223 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1000 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1047 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1012 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1019 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1050 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1149 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17618 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1029 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3102 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 30484 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1045 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 15826 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 14897 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1077 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4531 : OK
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 4311 : OK
     [exec] Zookeeper_readOnly::testReadOnly : elapsed 4085 : OK
     [exec] OK (74)
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] *** Error in `./zktest-mt': corrupted double-linked list: 0x000000000156c810 ***
     [exec] /bin/bash: line 5: 18590 Aborted                 ZKROOT=/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/../.. CLASSPATH=$CLASSPATH:$CLOVER_HOME/lib/clover.jar ${dir}$tst
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 17 minutes 24 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3615 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3615/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 65.20 MB...]
     [exec] Log Message Received: [2017-11-19 23:29:32,522:1605(0x2b139d98d540):ZOO_INFO@zookeeper_init_internal@1152: Initiating client connection, host=127.0.0.1:22181 sessionTimeout=10000 watcher=0x461d20 sessionId=0 sessionPasswd=<null> context=0x7ffe8a941dd0 flags=0]
     [exec] Log Message Received: [2017-11-19 23:29:32,523:1605(0x2b139f9ee700):ZOO_INFO@check_events@2425: initiated connection to server [127.0.0.1:22181]]
     [exec] Log Message Received: [2017-11-19 23:29:32,526:1605(0x2b139f9ee700):ZOO_INFO@check_events@2477: session establishment complete on server [127.0.0.1:22181], sessionId=0x1002e59b71c000f, negotiated timeout=10000 ]
     [exec]  : elapsed 1000 : OK
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 10180 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1026 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1003 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1007 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1013 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1039 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17177 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1012 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3028 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 30391 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1020 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 14630 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 14637 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1034 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4512 : OK
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 4245 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4047
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (13719) - No such process
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 14 minutes 56 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3614 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3614/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 57.35 MB...]
    [junit] 2017-11-18 23:26:34,817 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal
    [junit] 2017-11-18 23:26:34,817 [myid:] - INFO  [main:ClientBase@601] - tearDown starting
    [junit] 2017-11-18 23:26:34,817 [myid:] - INFO  [main:ClientBase@571] - STOPPING server
    [junit] 2017-11-18 23:26:34,817 [myid:] - INFO  [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:22240
    [junit] 2017-11-18 23:26:34,817 [myid:127.0.0.1:22058] - INFO  [main-SendThread(127.0.0.1:22058):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:22058. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-18 23:26:34,818 [myid:127.0.0.1:22058] - WARN  [main-SendThread(127.0.0.1:22058):ClientCnxn$SendThread@1235] - Session 0x107fb1f43710000 for server 127.0.0.1/127.0.0.1:22058, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-11-18 23:26:34,826 [myid:] - INFO  [main:ZooKeeperServer@545] - shutting down
    [junit] 2017-11-18 23:26:34,826 [myid:] - ERROR [main:ZooKeeperServer@509] - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
    [junit] 2017-11-18 23:26:34,826 [myid:] - INFO  [main:SessionTrackerImpl@232] - Shutting down
    [junit] 2017-11-18 23:26:34,827 [myid:] - INFO  [main:PrepRequestProcessor@1010] - Shutting down
    [junit] 2017-11-18 23:26:34,827 [myid:] - INFO  [main:SyncRequestProcessor@191] - Shutting down
    [junit] 2017-11-18 23:26:34,827 [myid:] - INFO  [ProcessThread(sid:0 cport:22240)::PrepRequestProcessor@155] - PrepRequestProcessor exited loop!
    [junit] 2017-11-18 23:26:34,827 [myid:] - INFO  [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
    [junit] 2017-11-18 23:26:34,827 [myid:] - INFO  [main:FinalRequestProcessor@481] - shutdown of request processor complete
    [junit] 2017-11-18 23:26:34,828 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port22240,name1=InMemoryDataTree]
    [junit] 2017-11-18 23:26:34,828 [myid:] - INFO  [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port22240]
    [junit] 2017-11-18 23:26:34,828 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 22240
    [junit] 2017-11-18 23:26:34,829 [myid:] - INFO  [main:JMXEnv@146] - ensureOnly:[]
    [junit] 2017-11-18 23:26:34,848 [myid:] - INFO  [main:ClientBase@626] - fdcount after test is: 7150 at start it was 7150
    [junit] 2017-11-18 23:26:34,848 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal
    [junit] 2017-11-18 23:26:34,849 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal
    [junit] Tests run: 105, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 408.54 sec, Thread: 5, Class: org.apache.zookeeper.test.NioNettySuiteTest
    [junit] 2017-11-18 23:26:35,255 [myid:127.0.0.1:22123] - INFO  [main-SendThread(127.0.0.1:22123):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:22123. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-18 23:26:35,256 [myid:127.0.0.1:22123] - WARN  [main-SendThread(127.0.0.1:22123):ClientCnxn$SendThread@1235] - Session 0x307fb2175ef0000 for server 127.0.0.1/127.0.0.1:22123, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1267: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1271: Tests failed!

Total time: 10 minutes 3 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
1 tests failed.
FAILED:  org.apache.zookeeper.test.FourLetterWordsQuorumTest.testFourLetterWords

Error Message:
Exception while executing four letter word: srst

Stack Trace:
java.io.IOException: Exception while executing four letter word: srst
	at org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:126)
	at org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:56)
	at org.apache.zookeeper.test.FourLetterWordsQuorumTest.verify(FourLetterWordsQuorumTest.java:109)
	at org.apache.zookeeper.test.FourLetterWordsQuorumTest.testFourLetterWords(FourLetterWordsQuorumTest.java:94)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:154)
	at java.io.BufferedReader.readLine(BufferedReader.java:317)
	at java.io.BufferedReader.readLine(BufferedReader.java:382)
	at org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:121)

ZooKeeper-trunk - Build # 3613 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3613/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 67.16 MB...]
     [exec] Log Message Received: [2017-11-16 23:30:27,271:32447(0x7fdd2979e700):ZOO_INFO@check_events@2477: session establishment complete on server [127.0.0.1:22181], sessionId=0x107f0dafe75000f, negotiated timeout=10000 ]
     [exec]  : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 10323 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1002 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1035 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1028 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1010 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1032 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1068 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17263 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1018 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 4062 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 32772 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1036 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 17151 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 15148 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1052 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4520 : OK
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 5390 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4018
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] Makefile:1744: recipe for target 'check-TESTS' failed
     [exec] make[1]: Leaving directory '/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] Makefile:2000: recipe for target 'check-am' failed
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (10156) - No such process
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 14 minutes 52 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3612 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3612/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 58.10 MB...]
    [junit] 2017-11-16 00:27:10,915 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.5]
    [junit] 2017-11-16 00:27:10,915 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.1]
    [junit] 2017-11-16 00:27:10,915 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.2]
    [junit] 2017-11-16 00:27:10,915 [myid:] - INFO  [QuorumPeer[myid=5](plain=/127.0.0.1:24848)(secure=disabled):MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id5,name1=replica.3]
    [junit] 2017-11-16 00:27:10,916 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 24836
    [junit] 2017-11-16 00:27:10,916 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24836 is no longer accepting client connections
    [junit] 2017-11-16 00:27:10,917 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 24839
    [junit] 2017-11-16 00:27:10,917 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24839 is no longer accepting client connections
    [junit] 2017-11-16 00:27:10,917 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 24842
    [junit] 2017-11-16 00:27:10,918 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24842 is no longer accepting client connections
    [junit] 2017-11-16 00:27:10,918 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 24845
    [junit] 2017-11-16 00:27:10,918 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24845 is no longer accepting client connections
    [junit] 2017-11-16 00:27:10,919 [myid:] - INFO  [main:FourLetterWordMain@87] - connecting to 127.0.0.1 24848
    [junit] 2017-11-16 00:27:10,919 [myid:] - INFO  [main:QuorumUtil@243] - 127.0.0.1:24848 is no longer accepting client connections
    [junit] 2017-11-16 00:27:10,921 [myid:] - INFO  [main:ZKTestCase$1@68] - SUCCEEDED testRemoveOneAsynchronous
    [junit] 2017-11-16 00:27:10,921 [myid:] - INFO  [main:ZKTestCase$1@63] - FINISHED testRemoveOneAsynchronous
    [junit] Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 347.332 sec, Thread: 6, Class: org.apache.zookeeper.test.ReconfigTest
    [junit] 2017-11-16 00:27:11,205 [myid:127.0.0.1:24771] - INFO  [main-SendThread(127.0.0.1:24771):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24771. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-16 00:27:11,207 [myid:127.0.0.1:24771] - WARN  [main-SendThread(127.0.0.1:24771):ClientCnxn$SendThread@1235] - Session 0x207ebe6aa370001 for server 127.0.0.1/127.0.0.1:24771, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-11-16 00:27:11,214 [myid:127.0.0.1:24738] - INFO  [main-SendThread(127.0.0.1:24738):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:24738. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-16 00:27:11,214 [myid:127.0.0.1:24738] - WARN  [main-SendThread(127.0.0.1:24738):ClientCnxn$SendThread@1235] - Session 0x207ebe61edd0000 for server 127.0.0.1/127.0.0.1:24738, unexpected error, closing socket connection and attempting reconnect
    [junit] java.net.ConnectException: Connection refused
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
    [junit] 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
    [junit] 2017-11-16 00:27:11,223 [myid:] - INFO  [SessionTracker:SessionTrackerImpl@158] - SessionTrackerImpl exited loop!
    [junit] Test org.apache.zookeeper.test.ReconfigTest FAILED

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1267: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1271: Tests failed!

Total time: 10 minutes 40 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2931
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
27 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest

Error Message:
Server 2 is not up

Stack Trace:
junit.framework.AssertionFailedError: Server 2 is not up
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startServer(StandaloneDisabledTest.java:175)
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startFollowers(StandaloneDisabledTest.java:206)
	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest(StandaloneDisabledTest.java:66)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncTest.testAsync

Error Message:
waiting for server down

Stack Trace:
junit.framework.AssertionFailedError: waiting for server down
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:374)
	at org.apache.zookeeper.test.AsyncTest.tearDown(AsyncTest.java:57)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous

Error Message:
Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNetty

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNetty
	at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:2936)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:879)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:799)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:939)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:686)
	at org.apache.zookeeper.admin.ZooKeeperAdmin.<init>(ZooKeeperAdmin.java:86)
	at org.apache.zookeeper.test.ReconfigTest.createAdminHandles(ReconfigTest.java:228)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:475)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:2933)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:151)
	at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:116)
	at org.apache.zookeeper.ClientCnxnSocketNetty.<init>(ClientCnxnSocketNetty.java:71)


FAILED:  org.apache.zookeeper.test.ACLTest.testAcls

Error Message:
Failed to bind to: 0.0.0.0/0.0.0.0:19301

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:19301
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:507)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ACLTest.testAcls(ACLTest.java:106)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ACLTest.testNullValueACL

Error Message:
Failed to bind to: 0.0.0.0/0.0.0.0:19301

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:19301
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:507)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ACLTest.testNullValueACL(ACLTest.java:244)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ACLTest.testDisconnectedAddAuth

Error Message:
Failed to bind to: 0.0.0.0/0.0.0.0:19301

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:19301
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:507)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ACLTest.testDisconnectedAddAuth(ACLTest.java:72)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ACLTest.testNullACL

Error Message:
Failed to bind to: 0.0.0.0/0.0.0.0:19301

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:19301
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:507)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ACLTest.testNullACL(ACLTest.java:202)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testBulkReconfig

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
	at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
	at org.apache.zookeeper.test.ReconfigTest.testBulkReconfig(ReconfigTest.java:434)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange

Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.lang.RuntimeException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange(ReconfigTest.java:991)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
	at org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:471)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeout

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.SessionTest.testMinMaxSessionTimeout

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionMove

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeoutAccess

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.SessionTest.testSession

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionStateNoDupStateReporting

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending

Error Message:
Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
	java.lang.OutOfMemoryError: unable to create new native thread
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:350)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
	at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
	at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1340)
	at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:180)
	at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:172)
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
	at sun.rmi.transport.Transport$2.run(Transport.java:202)
	at sun.rmi.transport.Transport$2.run(Transport.java:199)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:198)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.java:619)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:684)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:681)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherCorrectness

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithLocal

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithGlobal

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherCount

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithGlobal

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal

Error Message:
Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory

Stack Trace:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148)
	at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:133)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
	at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
	at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
	at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:329)


FAILED:  org.apache.zookeeper.test.ReconfigExceptionTest.testReconfigDisabled

Error Message:
Waiting for server down

Stack Trace:
junit.framework.AssertionFailedError: Waiting for server down
	at org.apache.zookeeper.test.QuorumUtil.shutdownAll(QuorumUtil.java:241)
	at org.apache.zookeeper.test.QuorumUtil.tearDown(QuorumUtil.java:306)
	at org.apache.zookeeper.test.ReconfigExceptionTest.tearDown(ReconfigExceptionTest.java:80)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower

Error Message:
client could not connect to reestablished quorum: giving up after 30+ seconds.

Stack Trace:
junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
	at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:183)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:795)
	at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower(ReconfigTest.java:714)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.SSLAuthTest.testMisconfiguration

Error Message:
Failed to bind to: localhost/127.0.0.1:21997

Stack Trace:
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:21997
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.start(NettyServerCnxnFactory.java:488)
	at org.apache.zookeeper.server.NettyServerCnxnFactory.startup(NettyServerCnxnFactory.java:507)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.test.ClientBase.startServerInstance(ClientBase.java:421)
	at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:527)
	at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518)
	at org.apache.zookeeper.test.SSLAuthTest.setUp(SSLAuthTest.java:55)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.zookeeper.test.ZkDatabaseCorruptionTest.testCorruption

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
	at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
	at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:155)
	at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:871)
	at org.apache.zookeeper.test.QuorumBase.setupServer(QuorumBase.java:356)
	at org.apache.zookeeper.test.QuorumBase.setupServers(QuorumBase.java:294)
	at org.apache.zookeeper.test.ZkDatabaseCorruptionTest.testCorruption(ZkDatabaseCorruptionTest.java:113)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)

ZooKeeper-trunk - Build # 3611 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3611/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 65.37 MB...]
     [exec] Log Message Received: [2017-11-15 23:29:26,859:18634(0x7f95e683e700):ZOO_INFO@check_events@2477: session establishment complete on server [127.0.0.1:22181], sessionId=0x107ebb3b42a000f, negotiated timeout=10000 ]
     [exec]  : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 11498 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1187 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1012 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1023 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1056 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1109 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17331 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1028 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3058 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 33299 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1097 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 18361 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 19347 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1145 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4555 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 5742 : OK
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (28989) - No such process
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4022
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] Makefile:1744: recipe for target 'check-TESTS' failed
     [exec] make[1]: Leaving directory '/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] Makefile:2000: recipe for target 'check-am' failed
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 15 minutes 4 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3610 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3610/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 65.89 MB...]
     [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server started : elapsed 11507 : OK
     [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
     [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
     [exec] Zookeeper_simpleSystem::testNullData : elapsed 1097 : OK
     [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1052 : OK
     [exec] Zookeeper_simpleSystem::testCreate : elapsed 1013 : OK
     [exec] Zookeeper_simpleSystem::testPath : elapsed 1056 : OK
     [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1134 : OK
     [exec] Zookeeper_simpleSystem::testPing : elapsed 17332 : OK
     [exec] Zookeeper_simpleSystem::testAcl : elapsed 1016 : OK
     [exec] Zookeeper_simpleSystem::testChroot : elapsed 3065 : OK
     [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper server started : elapsed 31553 : OK
     [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1031 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 17300 : OK
     [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper server started ZooKeeper server started ZooKeeper server started : elapsed 18396 : OK
     [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1111 : OK
     [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4529 : OK
     [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started : elapsed 5470 : OK
     [exec] rm: cannot remove '/tmp/zkdata/myid': Permission denied
     [exec] Zookeeper_readOnly::testReadOnly : assertion : elapsed 4015
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/TestReadOnlyClient.cc:99: Assertion: equality assertion failed [Expected: 0, Actual  : -4]
     [exec] Failures !!!
     [exec] Run: 74   Failure total: 1   Failures: 1   Errors: 0
     [exec] FAIL: zktest-mt
     [exec] ==========================================
     [exec] 1 of 2 tests failed
     [exec] Please report to user@zookeeper.apache.org
     [exec] ==========================================
     [exec] Makefile:1744: recipe for target 'check-TESTS' failed
     [exec] make[1]: Leaving directory '/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build/test/test-cppunit'
     [exec] Makefile:2000: recipe for target 'check-am' failed
     [exec] /home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/src/c/tests/zkServer.sh: line 62: kill: (777) - No such process
     [exec] make[1]: *** [check-TESTS] Error 1
     [exec] make: *** [check-am] Error 2

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1346: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1356: exec returned: 2

Total time: 15 minutes 22 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2923
[JIRA] Updating issue ZOOKEEPER-2934
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
All tests passed

ZooKeeper-trunk - Build # 3609 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See https://builds.apache.org/job/ZooKeeper-trunk/3609/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE ###########################
[...truncated 66.57 MB...]
    [junit] 2017-11-15 21:28:48,360 [myid:127.0.0.1:19427] - INFO  [main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1231] - channel for sessionid 0x200194bfe300000 is lost, closing socket connection and attempting reconnect
    [junit] 2017-11-15 21:28:48,799 [myid:127.0.0.1:19430] - INFO  [main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:19430. Will not attempt to authenticate using SASL (unknown error)
    [junit] 2017-11-15 21:28:48,799 [myid:] - INFO  [New I/O boss #10329:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19430
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-11-15 21:28:48,800 [myid:] - WARN  [New I/O boss #10329:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 0x8ae9e41d] EXCEPTION: java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19430
    [junit] java.net.ConnectException: Connection refused: 127.0.0.1/127.0.0.1:19430
    [junit] 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit] 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit] 	at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [junit] 	at java.lang.Thread.run(Thread.java:745)
    [junit] 2017-11-15 21:28:48,800 [myid:] - INFO  [New I/O boss #10329:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2017-11-15 21:28:48,800 [myid:127.0.0.1:19430] - INFO  [main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1231] - channel for sessionid 0x300194bfe8a0000 is lost, closing socket connection and attempting reconnect

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1386: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1267: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1271: Tests failed!

Total time: 12 minutes 2 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[JIRA] Updating issue ZOOKEEPER-2690
[JIRA] Updating issue ZOOKEEPER-2574
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###################################################################################
############################## FAILED TESTS (if any) ##############################
1 tests failed.
FAILED:  org.apache.zookeeper.server.quorum.Zab1_0Test.testNormalFollowerRunWithDiff

Error Message:
expected:<4294967298> but was:<0>

Stack Trace:
junit.framework.AssertionFailedError: expected:<4294967298> but was:<0>
	at org.apache.zookeeper.server.quorum.Zab1_0Test$5.converseWithFollower(Zab1_0Test.java:869)
	at org.apache.zookeeper.server.quorum.Zab1_0Test.testFollowerConversation(Zab1_0Test.java:517)
	at org.apache.zookeeper.server.quorum.Zab1_0Test.testNormalFollowerRunWithDiff(Zab1_0Test.java:784)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)