You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/03/09 09:18:13 UTC

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.6.0) - Build # 302 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/302/
Java: 64bit/jdk1.6.0 -XX:+UseParallelGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ClusterStateUpdateTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest:     1) Thread[id=4982, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[DDACD73892A7D07]-EventThread, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)         at org.apache.solr.cloud.ZkController.register(ZkController.java:766)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 
   1) Thread[id=4982, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[DDACD73892A7D07]-EventThread, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)
        at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)
        at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:766)
        at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
	at __randomizedtesting.SeedInfo.seed([DDACD73892A7D07]:0)




Build Log:
[...truncated 8548 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ClusterStateUpdateTest
[junit4:junit4]   2> 4 T4969 oas.SolrTestCaseJ4.setUp ###Starting testCoreRegistration
[junit4:junit4]   2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585
[junit4:junit4]   2> 5 T4969 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 6 T4970 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 7 T4970 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 7 T4970 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 8 T4970 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 8 T4970 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 10 T4970 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 107 T4969 oasc.ZkTestServer.run start zk server on port:50741
[junit4:junit4]   2> 108 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3577350f
[junit4:junit4]   2> 109 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T4975 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 110 T4975 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 110 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50742
[junit4:junit4]   2> 111 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50742
[junit4:junit4]   2> 111 T4973 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 113 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70000 with negotiated timeout 10000 for client /127.0.0.1:50742
[junit4:junit4]   2> 113 T4975 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T4976 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3577350f name:ZooKeeperConnection Watcher:127.0.0.1:50741 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 114 T4969 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 118 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70000
[junit4:junit4]   2> 143 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50742 which had sessionid 0x13d4e2f2dc70000
[junit4:junit4]   2> 143 T4976 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 143 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70000 closed
[junit4:junit4]   2> 152 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@d00447e
[junit4:junit4]   2> 153 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 153 T4977 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 154 T4977 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 155 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50743
[junit4:junit4]   2> 155 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50743
[junit4:junit4]   2> 158 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70001 with negotiated timeout 10000 for client /127.0.0.1:50743
[junit4:junit4]   2> 159 T4977 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70001, negotiated timeout = 10000
[junit4:junit4]   2> 159 T4978 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d00447e name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 160 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 160 T4969 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 166 T4969 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 171 T4969 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 175 T4969 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 180 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 181 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 189 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 190 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 195 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 195 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 199 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 200 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 205 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 206 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 211 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 211 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 216 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 217 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 221 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 222 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 226 T4969 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 227 T4969 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 232 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70001
[junit4:junit4]   2> 233 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70001 closed
[junit4:junit4]   2> 233 T4978 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 234 T4969 oasc.ClusterStateUpdateTest.setUp ####SETUP_START testCoreRegistration
[junit4:junit4]   2> 233 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50743 which had sessionid 0x13d4e2f2dc70001
[junit4:junit4]   2> 238 T4969 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 238 T4969 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 239 T4969 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 239 T4969 oasc.CoreContainer.<init> New CoreContainer 1410592845
[junit4:junit4]   2> 240 T4969 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 240 T4969 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 275 T4969 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 276 T4969 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 276 T4969 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 277 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 277 T4969 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 277 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 278 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 278 T4969 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 279 T4969 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 279 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 296 T4969 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 317 T4969 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:50741/solr
[junit4:junit4]   2> 318 T4969 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 318 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@10ed078a
[junit4:junit4]   2> 319 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 320 T4979 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 321 T4979 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 322 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50744
[junit4:junit4]   2> 322 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50744
[junit4:junit4]   2> 324 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70002 with negotiated timeout 20000 for client /127.0.0.1:50744
[junit4:junit4]   2> 325 T4979 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70002, negotiated timeout = 20000
[junit4:junit4]   2> 325 T4980 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10ed078a name:ZooKeeperConnection Watcher:127.0.0.1:50741 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 326 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 327 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70002
[junit4:junit4]   2> 328 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50744 which had sessionid 0x13d4e2f2dc70002
[junit4:junit4]   2> 329 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70002 closed
[junit4:junit4]   2> 329 T4980 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 329 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 343 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@55a379c7
[junit4:junit4]   2> 344 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 344 T4981 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 346 T4981 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 346 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50745
[junit4:junit4]   2> 346 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50745
[junit4:junit4]   2> 348 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70003 with negotiated timeout 3000 for client /127.0.0.1:50745
[junit4:junit4]   2> 348 T4981 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70003, negotiated timeout = 3000
[junit4:junit4]   2> 348 T4982 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55a379c7 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 349 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 351 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 353 T4969 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 357 T4969 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 358 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:1661_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 359 T4969 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 364 T4969 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 372 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 372 T4969 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 376 T4969 oasc.Overseer.start Overseer (id=89313532155723779-127.0.0.1:1661_solr-n_0000000000) starting
[junit4:junit4]   2> 376 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 378 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 379 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 380 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 382 T4984 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 382 T4969 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 386 T4969 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 389 T4969 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 394 T4983 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 398 T4985 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 399 T4985 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 400 T4985 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 400 T4985 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 402 T4985 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 403 T4985 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 403 T4985 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 460 T4985 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 528 T4985 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 529 T4985 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 534 T4985 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1173 T4985 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1182 T4985 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1186 T4985 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1207 T4985 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1213 T4985 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1219 T4985 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1221 T4985 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1221 T4985 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1221 T4985 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1223 T4985 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1224 T4985 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1224 T4985 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/
[junit4:junit4]   2> 1224 T4985 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24c68a98
[junit4:junit4]   2> 1225 T4985 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1225 T4985 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1 forceNew: false
[junit4:junit4]   2> 1226 T4985 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/index/
[junit4:junit4]   2> 1226 T4985 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1227 T4985 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/index forceNew: false
[junit4:junit4]   2> 1228 T4985 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@76b45dd3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b0d5993)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1229 T4985 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1230 T4985 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1231 T4985 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1232 T4985 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1232 T4985 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1233 T4985 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1233 T4985 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1234 T4985 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1234 T4985 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1234 T4985 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1235 T4985 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1236 T4985 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1236 T4985 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1237 T4985 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1237 T4985 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1237 T4985 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1238 T4985 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1238 T4985 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1239 T4985 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1240 T4985 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1240 T4985 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1241 T4985 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1241 T4985 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1242 T4985 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1242 T4985 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1243 T4985 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1243 T4985 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1244 T4985 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1249 T4985 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1252 T4985 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1254 T4985 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1256 T4985 oass.SolrIndexSearcher.<init> Opening Searcher@3b2cd266 main
[junit4:junit4]   2> 1257 T4985 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1257 T4985 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1258 T4985 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1263 T4985 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1301 T4986 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b2cd266 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1307 T4985 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1307 T4985 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1308 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x56 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 1901 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1903 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 1903 T4983 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 1904 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x5e zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 1911 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2313 T4985 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2313 T4985 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:1661/solr collection:collection1 shard:shard1
[junit4:junit4]   2> 2314 T4985 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 2324 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:delete cxid:0x78 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 2325 T4985 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2326 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x79 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2329 T4985 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2329 T4985 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2330 T4985 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:1661/solr/collection1/
[junit4:junit4]   2> 2330 T4985 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 2337 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x83 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3419 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3430 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3485 T4985 oasc.ZkController.register We are http://127.0.0.1:1661/solr/collection1/ and leader is http://127.0.0.1:1661/solr/collection1/
[junit4:junit4]   2> 3485 T4985 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 3486 T4985 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3486 T4985 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3488 T4985 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3490 T4969 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3491 T4969 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3492 T4969 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3492 T4969 oasc.CoreContainer.<init> New CoreContainer 456963134
[junit4:junit4]   2> 3493 T4969 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3493 T4969 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3528 T4969 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 3529 T4969 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3529 T4969 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3530 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3530 T4969 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3531 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3531 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3531 T4969 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3532 T4969 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3532 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 3548 T4969 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3568 T4969 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:50741/solr
[junit4:junit4]   2> 3569 T4969 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 3569 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7464e6ed
[junit4:junit4]   2> 3570 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3571 T4988 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 3572 T4988 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 3573 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50746
[junit4:junit4]   2> 3573 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50746
[junit4:junit4]   2> 3574 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70004 with negotiated timeout 20000 for client /127.0.0.1:50746
[junit4:junit4]   2> 3574 T4988 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70004, negotiated timeout = 20000
[junit4:junit4]   2> 3575 T4989 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7464e6ed name:ZooKeeperConnection Watcher:127.0.0.1:50741 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3575 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3577 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70004
[junit4:junit4]   2> 3578 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50746 which had sessionid 0x13d4e2f2dc70004
[junit4:junit4]   2> 3578 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70004 closed
[junit4:junit4]   2> 3578 T4989 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 3579 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 3593 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1cd5e0b7
[junit4:junit4]   2> 3594 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3595 T4990 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 3596 T4990 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 3596 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50747
[junit4:junit4]   2> 3597 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50747
[junit4:junit4]   2> 3598 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70005 with negotiated timeout 3000 for client /127.0.0.1:50747
[junit4:junit4]   2> 3598 T4990 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70005, negotiated timeout = 3000
[junit4:junit4]   2> 3599 T4991 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cd5e0b7 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3599 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3600 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70005 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3602 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70005 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3605 T4969 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4615 T4969 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1662_solr
[junit4:junit4]   2> 4616 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70005 type:delete cxid:0xb zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:1662_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:1662_solr
[junit4:junit4]   2> 4617 T4969 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1662_solr
[junit4:junit4]   2> 4622 T4982 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4622 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4623 T4991 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4632 T4992 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4633 T4992 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4634 T4992 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4634 T4992 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4636 T4992 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4637 T4992 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4637 T4992 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4699 T4992 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4784 T4992 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4785 T4992 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4792 T4992 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4939 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4940 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 4947 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4947 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5413 T4992 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5423 T4992 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5427 T4992 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5448 T4992 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5453 T4992 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5458 T4992 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5460 T4992 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 5460 T4992 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 5460 T4992 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5462 T4992 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 5462 T4992 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 5462 T4992 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/
[junit4:junit4]   2> 5463 T4992 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24c68a98
[junit4:junit4]   2> 5463 T4992 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 5464 T4992 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2 forceNew: false
[junit4:junit4]   2> 5464 T4992 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/index/
[junit4:junit4]   2> 5465 T4992 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5465 T4992 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/index forceNew: false
[junit4:junit4]   2> 5467 T4992 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@544849da lockFactory=org.apache.lucene.store.NativeFSLockFactory@201b0320)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5467 T4992 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5468 T4992 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5470 T4992 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5471 T4992 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5471 T4992 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5472 T4992 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5472 T4992 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5473 T4992 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5473 T4992 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5474 T4992 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5475 T4992 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5475 T4992 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5476 T4992 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5476 T4992 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5477 T4992 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5477 T4992 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5478 T4992 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5478 T4992 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5479 T4992 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5480 T4992 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5480 T4992 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5481 T4992 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5482 T4992 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5483 T4992 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5484 T4992 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5484 T4992 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5485 T4992 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5485 T4992 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5490 T4992 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5493 T4992 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5495 T4992 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5498 T4992 oass.SolrIndexSearcher.<init> Opening Searcher@2acdf4 main
[junit4:junit4]   2> 5498 T4992 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5499 T4992 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5499 T4992 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5504 T4992 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5533 T4993 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2acdf4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5541 T4992 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5541 T4992 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6456 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6457 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1662_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1662/solr"}
[junit4:junit4]   2> 6458 T4983 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 6458 T4983 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6465 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6465 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6545 T4992 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6545 T4992 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:1662/solr collection:collection1 shard:shard1
[junit4:junit4]   2> 6550 T4992 oasc.ZkController.register We are http://127.0.0.1:1662/solr/collection1/ and leader is http://127.0.0.1:1661/solr/collection1/
[junit4:junit4]   2> 6551 T4992 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 6551 T4992 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6551 T4992 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6553 T4992 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6555 T4969 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6555 T4969 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 6556 T4969 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 6556 T4969 oasc.CoreContainer.<init> New CoreContainer 1471121046
[junit4:junit4]   2> 6556 T4969 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6556 T4969 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6591 T4969 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 6591 T4969 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6591 T4969 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 6592 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6592 T4969 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6592 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6592 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6593 T4969 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6593 T4969 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6593 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 6611 T4969 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6627 T4969 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:50741/solr
[junit4:junit4]   2> 6628 T4969 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 6628 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3da861de
[junit4:junit4]   2> 6629 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6629 T4995 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 6630 T4995 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 6630 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50748
[junit4:junit4]   2> 6631 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50748
[junit4:junit4]   2> 6632 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70006 with negotiated timeout 20000 for client /127.0.0.1:50748
[junit4:junit4]   2> 6632 T4995 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70006, negotiated timeout = 20000
[junit4:junit4]   2> 6632 T4996 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3da861de name:ZooKeeperConnection Watcher:127.0.0.1:50741 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6633 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6634 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70006
[junit4:junit4]   2> 6635 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50748 which had sessionid 0x13d4e2f2dc70006
[junit4:junit4]   2> 6635 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70006 closed
[junit4:junit4]   2> 6636 T4996 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 6636 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 6642 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@30a5a549
[junit4:junit4]   2> 6643 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6643 T4997 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 6644 T4997 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 6645 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50749
[junit4:junit4]   2> 6645 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50749
[junit4:junit4]   2> 6646 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70007 with negotiated timeout 3000 for client /127.0.0.1:50749
[junit4:junit4]   2> 6646 T4997 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70007, negotiated timeout = 3000
[junit4:junit4]   2> 6647 T4998 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30a5a549 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6647 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6648 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70007 type:create cxid:0x1 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6649 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70007 type:create cxid:0x2 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6652 T4969 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7657 T4969 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1663_solr
[junit4:junit4]   2> 7658 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70007 type:delete cxid:0xb zxid:0x5e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:1663_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:1663_solr
[junit4:junit4]   2> 7659 T4969 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1663_solr
[junit4:junit4]   2> 7664 T4998 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7664 T4991 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7664 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7664 T4982 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7665 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7675 T4999 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 7675 T4999 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7676 T4999 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7677 T4999 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7681 T4999 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 7682 T4999 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7683 T4999 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 7736 T4999 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7805 T4999 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7806 T4999 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7812 T4999 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7972 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7973 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1662_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1662/solr"}
[junit4:junit4]   2> 7979 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7981 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7981 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8430 T4999 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8439 T4999 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8443 T4999 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8464 T4999 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8470 T4999 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8475 T4999 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8477 T4999 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8478 T4999 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8478 T4999 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8480 T4999 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8480 T4999 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8480 T4999 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/
[junit4:junit4]   2> 8481 T4999 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24c68a98
[junit4:junit4]   2> 8481 T4999 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 8482 T4999 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3 forceNew: false
[junit4:junit4]   2> 8482 T4999 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/index/
[junit4:junit4]   2> 8483 T4999 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8483 T4999 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/index forceNew: false
[junit4:junit4]   2> 8485 T4999 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@9e9efec lockFactory=org.apache.lucene.store.NativeFSLockFactory@262c2ad7)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8485 T4999 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8486 T4999 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8488 T4999 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8488 T4999 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8489 T4999 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8489 T4999 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8490 T4999 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8490 T4999 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8491 T4999 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8491 T4999 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8492 T4999 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8493 T4999 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8493 T4999 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8494 T4999 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8494 T4999 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8494 T4999 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8495 T4999 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8496 T4999 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8496 T4999 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8497 T4999 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8498 T4999 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8498 T4999 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8499 T4999 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8501 T4999 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8501 T4999 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8502 T4999 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8503 T4999 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8504 T4999 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8510 T4999 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8514 T4999 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8515 T4999 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8518 T4999 oass.SolrIndexSearcher.<init> Opening Searcher@1c7f2dbf main
[junit4:junit4]   2> 8518 T4999 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8518 T4999 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8519 T4999 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8524 T4999 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8551 T5000 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c7f2dbf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8559 T4999 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8559 T4999 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9488 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9489 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1663_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1663/solr"}
[junit4:junit4]   2> 9490 T4983 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 9490 T4983 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9497 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9497 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9497 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9563 T4999 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9564 T4999 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:1663/solr collection:collection1 shard:shard1
[junit4:junit4]   2> 9570 T4999 oasc.ZkController.register We are http://127.0.0.1:1663/solr/collection1/ and leader is http://127.0.0.1:1661/solr/collection1/
[junit4:junit4]   2> 9570 T4999 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 9570 T4999 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9571 T4999 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9573 T4999 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9576 T4969 oasc.ClusterStateUpdateTest.setUp ####SETUP_END testCoreRegistration
[junit4:junit4]   2> 9577 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@30075a92
[junit4:junit4]   2> 9578 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9578 T5002 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 9580 T5002 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 9580 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50750
[junit4:junit4]   2> 9581 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50750
[junit4:junit4]   2> 9583 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70008 with negotiated timeout 10000 for client /127.0.0.1:50750
[junit4:junit4]   2> 9583 T5002 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70008, negotiated timeout = 10000
[junit4:junit4]   2> 9583 T5003 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30075a92 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9584 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9584 T4969 oascc.SolrZkClient.makePath makePath: /collections/testcore
[junit4:junit4]   2> 9589 T4969 oascc.SolrZkClient.makePath makePath: /collections/testcore/shards
[junit4:junit4]   2> 9594 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70008
[junit4:junit4]   2> 9595 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50750 which had sessionid 0x13d4e2f2dc70008
[junit4:junit4]   2> 9595 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70008 closed
[junit4:junit4]   2> 9595 T5003 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9596 T4969 oasc.CoreContainer.create Creating SolrCore 'testcore' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/testcore
[junit4:junit4]   2> 9597 T4969 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testcore
[junit4:junit4]   2> 9598 T4969 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9599 T4969 oasc.ZkController.readConfigName Load collection config from:/collections/testcore
[junit4:junit4]   2> 9602 T4969 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/testcore/'
[junit4:junit4]   2> 9645 T4969 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9708 T4969 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9709 T4969 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9714 T4969 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10324 T4969 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10333 T4969 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10337 T4969 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10360 T4969 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10366 T4969 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10371 T4969 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10373 T4969 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10374 T4969 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10375 T4969 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10377 T4969 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10377 T4969 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10378 T4969 oasc.SolrCore.<init> [testcore] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/testcore/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/
[junit4:junit4]   2> 10378 T4969 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24c68a98
[junit4:junit4]   2> 10379 T4969 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 10380 T4969 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4 forceNew: false
[junit4:junit4]   2> 10380 T4969 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/index/
[junit4:junit4]   2> 10381 T4969 oasc.SolrCore.initIndex WARNING [testcore] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10382 T4969 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/index forceNew: false
[junit4:junit4]   2> 10384 T4969 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b6f333d lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fffa6be)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10384 T4969 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10385 T4969 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10387 T4969 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10388 T4969 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10388 T4969 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10389 T4969 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10390 T4969 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10390 T4969 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10391 T4969 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10392 T4969 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10392 T4969 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10393 T4969 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10394 T4969 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10395 T4969 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10395 T4969 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10396 T4969 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10396 T4969 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10397 T4969 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10398 T4969 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10398 T4969 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10399 T4969 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10400 T4969 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10400 T4969 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10401 T4969 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10403 T4969 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10404 T4969 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10404 T4969 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10405 T4969 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10410 T4969 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10413 T4969 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10415 T4969 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10417 T4969 oass.SolrIndexSearcher.<init> Opening Searcher@52a0251f main
[junit4:junit4]   2> 10418 T4969 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10418 T4969 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10419 T4969 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10425 T4969 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10456 T5004 oasc.SolrCore.registerSearcher [testcore] Registered new searcher Searcher@52a0251f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10461 T4969 oasc.ZkController.publish publishing core=testcore state=down
[junit4:junit4]   2> 10462 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11006 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11007 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1663_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1663/solr"}
[junit4:junit4]   2> 11013 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"testcore",
[junit4:junit4]   2> 	  "collection":"testcore",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 11013 T4983 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11019 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11020 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11020 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11466 T4969 oasc.CoreContainer.registerCore registering core: testcore
[junit4:junit4]   2> 11466 T4969 oasc.ZkController.register Register replica - core:testcore address:http://127.0.0.1:1661/solr collection:testcore shard:shard1
[junit4:junit4]   2> 11468 T4969 oascc.SolrZkClient.makePath makePath: /collections/testcore/leader_elect/shard1/election
[junit4:junit4]   2> 11479 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:delete cxid:0x136 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/testcore/leaders Error:KeeperErrorCode = NoNode for /solr/collections/testcore/leaders
[junit4:junit4]   2> 11480 T4969 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 11481 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x137 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11484 T4969 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 11484 T4969 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 11484 T4969 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:1661/solr/testcore/
[junit4:junit4]   2> 11485 T4969 oascc.SolrZkClient.makePath makePath: /collections/testcore/leaders/shard1
[junit4:junit4]   2> 11491 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc70003 type:create cxid:0x141 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12531 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12543 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12543 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12543 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12567 T4969 oasc.ZkController.register We are http://127.0.0.1:1661/solr/testcore/ and leader is http://127.0.0.1:1661/solr/testcore/
[junit4:junit4]   2> 12568 T4969 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 12568 T4969 oasc.ZkController.publish publishing core=testcore state=active
[junit4:junit4]   2> 12568 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12571 T4969 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12572 T4969 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1471121046
[junit4:junit4]   2> 14054 T4983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14056 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"testcore",
[junit4:junit4]   2> 	  "collection":"testcore",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 14061 T4983 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:1663_solr_collection1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1663_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1663/solr"}
[junit4:junit4]   2> 14068 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14068 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14068 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15577 T4969 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15578 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15583 T4969 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2c3a7aea
[junit4:junit4]   2> 15597 T4969 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 15597 T4969 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 15598 T4969 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 15598 T4969 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 15599 T4969 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 15599 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3
[junit4:junit4]   2> 15600 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3
[junit4:junit4]   2> 15601 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/index
[junit4:junit4]   2> 15601 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data3/index
[junit4:junit4]   2> 15602 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70007
[junit4:junit4]   2> 15604 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50749 which had sessionid 0x13d4e2f2dc70007
[junit4:junit4]   2> 15604 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70007 closed
[junit4:junit4]   2> 15604 T4998 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 15604 T4998 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15605 T4998 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 15605 T4982 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 15605 T4998 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15605 T4991 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 15605 T4982 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 15613 T4991 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 15805 T4991 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15805 T4969 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=456963134
[junit4:junit4]   2> 15815 T4971 oazs.NIOServerCnxn.doIO WARNING caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13d4e2f2dc70005, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 15815 T4969 oasc.CoreContainer.shutdown SEVERE  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:219)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:216)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:216)
[junit4:junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:271)
[junit4:junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:258)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.publishAndWaitForDownStates(ZkController.java:556)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:666)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration(ClusterStateUpdateTest.java:221)
[junit4:junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 15816 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50747 which had sessionid 0x13d4e2f2dc70005
[junit4:junit4]   2> 15817 T4969 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15818 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15818 T4969 oasc.CoreMaps.publishCoresAsDown SEVERE  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:219)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:216)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:216)
[junit4:junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:271)
[junit4:junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:258)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.publish(ZkController.java:997)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.publish(ZkController.java:966)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreMaps.publishCoresAsDown(CoreContainer.java:1675)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:677)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration(ClusterStateUpdateTest.java:221)
[junit4:junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 15827 T4969 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@52b156e1
[junit4:junit4]   2> 15828 T4969 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 15828 T4969 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 15829 T4969 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 15829 T4969 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 15829 T4969 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 15830 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/index
[junit4:junit4]   2> 15830 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2/index
[junit4:junit4]   2> 15831 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2
[junit4:junit4]   2> 15831 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data2
[junit4:junit4]   2> 15832 T4969 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15832 T4969 oasc.SolrResourceLoader.locateSolrHome solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4:junit4]   2> 15832 T4969 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/solr/solr.xml
[junit4:junit4]   2> 15833 T4969 oasc.CoreContainer.<init> New CoreContainer 689464361
[junit4:junit4]   2> 15834 T4969 oasc.CoreContainer$Initializer.initialize no solr.xml or solr.properties file found - using default old-style solr.xml
[junit4:junit4]   2> 15834 T4969 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 15834 T4969 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 15868 T4969 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 15869 T4969 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15869 T4969 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 15870 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15870 T4969 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15871 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15871 T4969 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15871 T4969 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15872 T4969 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15872 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 15884 T4969 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15901 T4969 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:50741/solr
[junit4:junit4]   2> 15901 T4969 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15902 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@65568d2
[junit4:junit4]   2> 15903 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15903 T5005 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 15904 T5005 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 15905 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50751
[junit4:junit4]   2> 15905 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50751
[junit4:junit4]   2> 15906 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc70009 with negotiated timeout 20000 for client /127.0.0.1:50751
[junit4:junit4]   2> 15907 T5005 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc70009, negotiated timeout = 20000
[junit4:junit4]   2> 15907 T5006 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65568d2 name:ZooKeeperConnection Watcher:127.0.0.1:50741 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15907 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15909 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70009
[junit4:junit4]   2> 15910 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50751 which had sessionid 0x13d4e2f2dc70009
[junit4:junit4]   2> 15910 T5006 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15910 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc70009 closed
[junit4:junit4]   2> 15913 T4969 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=0&connTimeout=0
[junit4:junit4]   2> 19425 T4972 oazs.ZooKeeperServer.expire Expiring session 0x13d4e2f2dc70005, timeout of 3000ms exceeded
[junit4:junit4]   2> 21116 T4969 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@2f6cbd0b
[junit4:junit4]   2> 21116 T4981 oaz.ClientCnxn$SendThread.run Client session timed out, have not heard from server in 5502ms for sessionid 0x13d4e2f2dc70003, closing socket connection and attempting reconnect
[junit4:junit4]   2> 21117 T4972 oazs.ZooKeeperServer.expire Expiring session 0x13d4e2f2dc70003, timeout of 3000ms exceeded
[junit4:junit4]   2> 21117 T4971 oazs.NIOServerCnxn.doIO WARNING caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13d4e2f2dc70003, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 21118 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50745 which had sessionid 0x13d4e2f2dc70003
[junit4:junit4]   2> 21117 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70005
[junit4:junit4]   2> 21118 T5007 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 21119 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc70003
[junit4:junit4]   2> 21119 T4969 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 21121 T5007 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 21121 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50752
[junit4:junit4]   2> 21122 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50752
[junit4:junit4]   2> 21123 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc7000a with negotiated timeout 3000 for client /127.0.0.1:50752
[junit4:junit4]   2> 21123 T5007 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc7000a, negotiated timeout = 3000
[junit4:junit4]   2> 21123 T5008 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f6cbd0b name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 21124 T4969 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 21124 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x1 zxid:0x9a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 21126 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x2 zxid:0x9b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 21129 T4969 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 21218 T4982 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55a379c7 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 21219 T4982 oascc.ConnectionManager.process zkClient has disconnected
[junit4:junit4]   2> 21219 T4982 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: null state: Disconnected type None
[junit4:junit4]   2> 22139 T4969 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/192.168.57.17:8983_solr
[junit4:junit4]   2> 22140 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:delete cxid:0xb zxid:0x9c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/192.168.57.17:8983_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/192.168.57.17:8983_solr
[junit4:junit4]   2> 22141 T4969 oascc.SolrZkClient.makePath makePath: /live_nodes/192.168.57.17:8983_solr
[junit4:junit4]   2> 22145 T5008 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4]   2> 22148 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:delete cxid:0x15 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 22149 T4969 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 22153 T4969 oasc.Overseer.start Overseer (id=89313532155723786-192.168.57.17:8983_solr-n_0000000003) starting
[junit4:junit4]   2> 22153 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x1a zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 22155 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x1b zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 22156 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x1c zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 22157 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000a type:create cxid:0x1d zxid:0xa4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 22159 T5010 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 22160 T5009 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 22162 T5011 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 22162 T5009 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22162 T5011 oasc.ZkController.createCollectionZkNode Check for collection zkNode:
[junit4:junit4]   2> 22163 T5011 oasc.CoreContainer.recordAndThrow SEVERE Unable to create core: collection1 java.lang.IllegalArgumentException: Path must not end with / character
[junit4:junit4]   2> 	at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1020)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:202)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:199)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:199)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.createCollectionZkNode(ZkController.java:1057)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:863)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:580)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:575)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 22163 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1663_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1663/solr"}
[junit4:junit4]   2> 22164 T5011 oasc.SolrException.log SEVERE null:org.apache.solr.common.SolrException: Unable to create core: collection1
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1356)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.create(CoreContainer.java:965)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:580)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:575)
[junit4:junit4]   2> 		at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4]   2> 		at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4]   2> 		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
[junit4:junit4]   2> 		at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4]   2> 		at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 	Caused by: java.lang.IllegalArgumentException: Path must not end with / character
[junit4:junit4]   2> 		at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58)
[junit4:junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1020)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:202)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:199)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:199)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.createCollectionZkNode(ZkController.java:1057)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:863)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
[junit4:junit4]   2> 		... 10 more
[junit4:junit4]   2> 	
[junit4:junit4]   2> 22170 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 23146 T4981 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 23147 T4981 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 23148 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50753
[junit4:junit4]   2> 23148 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to renew session 0x13d4e2f2dc70003 at /127.0.0.1:50753
[junit4:junit4]   2> 23148 T4971 oazs.ZooKeeperServer.finishSessionInit Invalid session 0x13d4e2f2dc70003 for client /127.0.0.1:50753, probably expired
[junit4:junit4]   2> 23149 T4981 oaz.ClientCnxn$SendThread.run Unable to reconnect to ZooKeeper service, session 0x13d4e2f2dc70003 has expired, closing socket connection
[junit4:junit4]   2> 23149 T4982 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55a379c7 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:Expired type:None path:null path:null type:None
[junit4:junit4]   2> 23149 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50753 which had sessionid 0x13d4e2f2dc70003
[junit4:junit4]   2> 23149 T4982 oascc.ConnectionManager.process Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4:junit4]   2> 23150 T4982 oascc.DefaultConnectionStrategy.reconnect Connection expired - starting a new one...
[junit4:junit4]   2> 23150 T4982 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:50741/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@55a379c7
[junit4:junit4]   2> 23151 T4982 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 23151 T5013 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server 127.0.0.1/127.0.0.1:50741. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 23152 T5013 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to 127.0.0.1/127.0.0.1:50741, initiating session
[junit4:junit4]   2> 23152 T4971 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:50754
[junit4:junit4]   2> 23153 T4971 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:50754
[junit4:junit4]   2> 23154 T4973 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4e2f2dc7000b with negotiated timeout 3000 for client /127.0.0.1:50754
[junit4:junit4]   2> 23154 T5013 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server 127.0.0.1/127.0.0.1:50741, sessionid = 0x13d4e2f2dc7000b, negotiated timeout = 3000
[junit4:junit4]   2> 23154 T5014 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55a379c7 name:ZooKeeperConnection Watcher:127.0.0.1:50741/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 23155 T4982 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 23155 T4982 oascc.ConnectionManager$1.update Connection with ZooKeeper reestablished.
[junit4:junit4]   2> 23156 T4982 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 23156 T4982 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 23163 T4982 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 23166 T4982 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 23167 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000b type:delete cxid:0xc zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:1661_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 23167 T4982 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1661_solr
[junit4:junit4]   2> 23170 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 23171 T5014 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 23171 T4982 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:1661/solr collection:collection1 shard:shard1
[junit4:junit4]   2> 23173 T5008 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 23174 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000b type:delete cxid:0x15 zxid:0xaf txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4]   2> 23175 T4982 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 23175 T4969 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1410592845
[junit4:junit4]   2> 23176 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000b type:create cxid:0x16 zxid:0xb0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 23178 T4982 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=3 found=1 timeoutin=180000
[junit4:junit4]   2> 23253 T4983 oasc.Overseer$ClusterStateUpdater.amILeader WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:253)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:250)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:250)
[junit4:junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:202)
[junit4:junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:125)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]   2> 
[junit4:junit4]   2> 23253 T4984 oasc.OverseerCollectionProcessor.run WARNING Overseer cannot talk to ZK
[junit4:junit4]   2> 23254 T4983 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89313532155723779-127.0.0.1:1661_solr-n_0000000000) am no longer a leader.
[junit4:junit4]   2> 23679 T5009 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23680 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 23687 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:1661_solr_collection1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 23699 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:1661_solr_testcore",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"testcore",
[junit4:junit4]   2> 	  "collection":"testcore",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 23707 T5014 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 23707 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 25180 T4969 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 25181 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 25183 T4969 oasc.ZkController.publish publishing core=testcore state=down
[junit4:junit4]   2> 25183 T4969 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 25186 T4982 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 25187 T4982 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:1661/solr/collection1/
[junit4:junit4]   2> 25187 T4969 oasc.SolrCore.close [testcore]  CLOSING SolrCore org.apache.solr.core.SolrCore@2b54907e
[junit4:junit4]   2> 25187 T4982 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5d77f519
[junit4:junit4]   2> 25189 T4982 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 25189 T4982 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 25190 T4982 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 25190 T4982 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 25190 T4982 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 25191 T4982 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/index
[junit4:junit4]   2> 25191 T4982 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1/index
[junit4:junit4]   2> 25191 T4982 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1
[junit4:junit4]   2> 25192 T4982 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data1
[junit4:junit4]   2> 25192 T4982 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 25197 T4974 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4e2f2dc7000b type:create cxid:0x28 zxid:0xc4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 25201 T4969 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 25202 T4969 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 25202 T4969 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 25203 T4969 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 25203 T4969 oasc.SolrCore.closeSearcher [testcore] Closing main searcher on request.
[junit4:junit4]   2> 25204 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/index
[junit4:junit4]   2> 25204 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4/index
[junit4:junit4]   2> 25204 T4969 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4
[junit4:junit4]   2> 25205 T4969 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1362816347585/data4
[junit4:junit4]   2> 25206 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc7000b
[junit4:junit4]   2> 25207 T5014 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 25208 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 25208 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50754 which had sessionid 0x13d4e2f2dc7000b
[junit4:junit4]   2> 25208 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc7000b closed
[junit4:junit4]   2> 25209 T4969 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=689464361
[junit4:junit4]   2> 25211 T5008 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4]   2> 25228 T5009 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25229 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 25234 T5009 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"testcore",
[junit4:junit4]   2> 	  "collection":"testcore",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:1661_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:1661/solr"}
[junit4:junit4]   2> 25247 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 26211 T5009 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89313532155723786-192.168.57.17:8983_solr-n_0000000003) am no longer a leader.
[junit4:junit4]   2> 26217 T4974 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4e2f2dc7000a
[junit4:junit4]   2> 26218 T4971 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:50752 which had sessionid 0x13d4e2f2dc7000a
[junit4:junit4]   2> 26220 T5008 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 26219 T4969 oaz.ZooKeeper.close Session: 0x13d4e2f2dc7000a closed
[junit4:junit4]   2> 26225 T5008 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 26226 T4969 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1471121046
[junit4:junit4]   2> 26226 T5008 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 26227 T5008 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 26509 T5014 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 26509 T5014 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 26510 T5014 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 27228 T4969 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4]   2> 27228 T4969 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4]   2> 27229 T4969 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 27229 T4974 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
[junit4:junit4]   2> 27229 T4969 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 27232 T4973 oazs.SyncRequestProcessor.run SyncRequestProcessor exited!
[junit4:junit4]   2> 27234 T4969 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 27423 T4972 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 28546 T4969 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50741 50741
[junit4:junit4]   2> 29368 T4971 oazs.NIOServerCnxnFactory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 29369 T4969 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4]   2> 29370 T4969 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4]   2> 29370 T4969 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 29371 T4969 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 29371 T4969 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 29371 T4969 oas.SolrTestCaseJ4.tearDown ###Ending testCoreRegistration
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 29373 T4969 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 29394 T4968 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 49411 T4968 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 
[junit4:junit4]   2> 	   1) Thread[id=4982, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[DDACD73892A7D07]-EventThread, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.register(ZkController.java:766)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 49411 T4968 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=4982, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[DDACD73892A7D07]-EventThread, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]   2> 49412 T4982 oasc.ZkController.getLeader SEVERE Error getting leader from zk java.lang.InterruptedException: sleep interrupted
[junit4:junit4]   2> 	at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.register(ZkController.java:766)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 49413 T4982 oasc.SolrException.log SEVERE Error registering SolrCore:org.apache.solr.common.SolrException: Error getting leader from zk
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:854)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.register(ZkController.java:766)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 	Caused by: java.lang.InterruptedException: sleep interrupted
[junit4:junit4]   2> 		at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)
[junit4:junit4]   2> 		... 7 more
[junit4:junit4]   2> 	
[junit4:junit4]   2> 49413 T4982 oasc.ZkController.register Register replica - core:testcore address:http://127.0.0.1:1661/solr collection:testcore shard:shard1
[junit4:junit4]   2> 49414 T4982 oasc.SolrException.log SEVERE Error registering SolrCore:org.apache.solr.common.cloud.ZooKeeperException: 
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.register(ZkController.java:757)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 	Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/testcore/leader_elect/shard1/election
[junit4:junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:202)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:199)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:199)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:92)
[junit4:junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:86)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElector.setup(LeaderElector.java:278)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:923)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.register(ZkController.java:751)
[junit4:junit4]   2> 		... 6 more
[junit4:junit4]   2> 	
[junit4:junit4]   2> 49414 T4982 oascc.DefaultConnectionStrategy.reconnect Reconnected to ZooKeeper
[junit4:junit4]   2> 49414 T4982 oascc.ConnectionManager.process Connected:true
[junit4:junit4]   2> 49415 T4982 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: null state: Expired type None
[junit4:junit4]   2> 49415 T4982 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 49415 T4968 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=DefaultSimilarity, locale=pt_PT, timezone=Asia/Dacca
[junit4:junit4]   2> NOTE: Mac OS X 10.8.2 x86_64/Apple Inc. 1.6.0_41 (64-bit)/cpus=2,threads=1,free=84870704,total=297205760
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SliceStateUpdateTest, PrimUtilsTest, TestXIncludeConfig, TestIndexSearcher, PeerSyncTest, TestCoreContainer, TestWordDelimiterFilterFactory, PolyFieldTest, SimpleFacetsTest, JsonLoaderTest, CurrencyFieldOpenExchangeTest, SearchHandlerTest, TestSolrQueryParser, TestConfig, PingRequestHandlerTest, ChaosMonkeySafeLeaderTest, TestPhraseSuggestions, SystemInfoHandlerTest, SpellPossibilityIteratorTest, QueryEqualityTest, RegexBoostProcessorTest, ResponseLogComponentTest, TestStressVersions, ShardRoutingTest, TestPHPSerializedResponseWriter, TestNumberUtils, NumericFieldsTest, TestDistributedSearch, SortByFunctionTest, TestIBSimilarityFactory, TestFiltering, TestFastWriter, TestAtomicUpdateErrorCases, TestUtils, CoreContainerCoreInitFailuresTest, TestCSVLoader, URLClassifyProcessorTest, SuggesterFSTTest, TestFieldResource, XsltUpdateRequestHandlerTest, AutoCommitTest, SolrCoreCheckLockOnStartupTest, SOLR749Test, BasicZkTest, StandardRequestHandlerTest, MoreLikeThisHandlerTest, TestRTGBase, NotRequiredUniqueKeyTest, TestCSVResponseWriter, FieldAnalysisRequestHandlerTest, SolrInfoMBeanTest, ZkCLITest, TestExtendedDismaxParser, TermsComponentTest, AliasIntegrationTest, MinimalSchemaTest, TestFastOutputStream, UUIDFieldTest, HighlighterTest, DistributedSpellCheckComponentTest, ShowFileRequestHandlerTest, MultiTermTest, TestDocumentBuilder, SampleTest, TestBM25SimilarityFactory, TestDynamicFieldResource, TestLMDirichletSimilarityFactory, SpellCheckCollatorTest, DisMaxRequestHandlerTest, RAMDirectoryFactoryTest, SynonymTokenizerTest, TestReload, TestRecovery, DateMathParserTest, UpdateRequestProcessorFactoryTest, TestBadConfig, ClusterStateTest, TestSolrCoreProperties, DebugComponentTest, TestPropInject, TestComponentsName, ScriptEngineTest, TimeZoneUtilsTest, DirectUpdateHandlerOptimizeTest, TestSearchPerf, TestBinaryResponseWriter, TestReplicationHandler, BasicFunctionalityTest, TestMultiCoreConfBootstrap, TestUpdate, SpatialFilterTest, SoftAutoCommitTest, QueryElevationComponentTest, TestDynamicFieldCollectionResource, TermVectorComponentDistributedTest, TestSuggestSpellingConverter, OpenCloseCoreStressTest, BinaryUpdateRequestHandlerTest, TestWriterPerf, TestSystemIdResolver, SolrIndexConfigTest, TestPropInjectDefaults, QueryParsingTest, TestPluginEnable, TestPostingsSolrHighlighter, TestQueryUtils, TestTrie, TestElisionMultitermQuery, RecoveryZkTest, TestSolrDeletionPolicy1, OutputWriterTest, CopyFieldTest, FullSolrCloudDistribCmdsTest, TestPartialUpdateDeduplication, TestFieldTypeResource, TestCharFilters, ReturnFieldsTest, TestQuerySenderNoQuery, WordBreakSolrSpellCheckerTest, SolrIndexSplitterTest, TestOmitPositions, DirectSolrConnectionTest, TestLFUCache, SchemaVersionSpecificBehaviorTest, LukeRequestHandlerTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestFaceting, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, OverseerTest, ClusterStateUpdateTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ClusterStateUpdateTest -Dtests.seed=DDACD73892A7D07 -Dtests.slow=true -Dtests.locale=pt_PT -Dtests.timezone=Asia/Dacca -Dtests.file.encoding=MacRoman
[junit4:junit4] ERROR   0.00s | ClusterStateUpdateTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 
[junit4:junit4]    >    1) Thread[id=4982, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[DDACD73892A7D07]-EventThread, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]    >         at java.lang.Thread.sleep(Native Method)
[junit4:junit4]    >         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:479)
[junit4:junit4]    >         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:455)
[junit4:junit4]    >         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:833)
[junit4:junit4]    >         at org.apache.solr.cloud.ZkController.register(ZkController.java:766)
[junit4:junit4]    >         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:212)
[junit4:junit4]    >         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
[junit4:junit4]    >         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
[junit4:junit4]    >         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
[junit4:junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([DDACD73892A7D07]:0)
[junit4:junit4] Completed in 49.46s, 1 test, 1 error <<< FAILURES!

[...truncated 399 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:381: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:361: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:183: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:449: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1213: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:877: There were test failures: 269 suites, 1137 tests, 1 suite-level error, 18 ignored (6 assumptions)

Total time: 80 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 64bit/jdk1.6.0 -XX:+UseParallelGC
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.6.0) - Build # 303 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/303/
Java: 64bit/jdk1.6.0 -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 8224 lines...]
[junit4:junit4] ERROR: JVM J0 ended with an exception, command line: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=7E51AFB20A353082 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/testlogging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp -Dclover.db.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Dfile.encoding=US-ASCII -classpath /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/test:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-test-framework/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/lucene-codecs-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/cglib-nodep-2.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-cli-1.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-codec-1.7.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-fileupload-1.2.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-lang-2.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/easymock-3.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/guava-13.0.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/javax.servlet-api-3.0.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/objenesis-1.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet-2.1.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/spatial4j-0.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/commons-io-2.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpclient-4.2.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpcore-4.2.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpmime-4.2.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-api-1.6.4.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/wstx-asl-3.2.7.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/zookeeper-3.4.5.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/servlet-api-3.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/core/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -flush -eventsfile /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20130309_125556_175.events @/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20130309_125556_175.suites
[junit4:junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 136. Very likely a JVM crash.  Process output piped in logs above.
[junit4:junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1253)
[junit4:junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:66)
[junit4:junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:821)
[junit4:junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:818)
[junit4:junit4] 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit4:junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit4:junit4] 	at java.lang.Thread.run(Thread.java:680)

BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:381: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:361: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:183: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:449: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1213: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:877: At least one slave process threw an exception, first: Forked process returned with error code: 136. Very likely a JVM crash.  Process output piped in logs above.

Total time: 41 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 64bit/jdk1.6.0 -XX:+UseConcMarkSweepGC
Email was triggered for: Failure
Sending email for trigger: Failure