You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/03/25 12:15:42 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build #
9901 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9901/
Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerRolesTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=1034, name=Overseer-91471531752816648-127.0.0.1:44816__fh-n_0000000010, state=RUNNABLE, group=Overseer collection creation process.]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1034, name=Overseer-91471531752816648-127.0.0.1:44816__fh-n_0000000010, state=RUNNABLE, group=Overseer collection creation process.]
at __randomizedtesting.SeedInfo.seed([F17FBEC930E6049:8EF175F4E4510075]:0)
Caused by: org.apache.solr.common.cloud.ZooKeeperException:
at __randomizedtesting.SeedInfo.seed([F17FBEC930E6049]:0)
at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:274)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000006
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:206)
at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:257)
... 1 more
Build Log:
[...truncated 10159 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
[junit4] 2> 241546 T857 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_fh/
[junit4] 2> 241549 T857 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1395744808238
[junit4] 2> 241550 T857 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 241550 T858 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 241650 T857 oasc.ZkTestServer.run start zk server on port:43489
[junit4] 2> 241651 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 241773 T864 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fe9875 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 241774 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 241774 T857 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 241778 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 241779 T866 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c2636f name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 241779 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 241779 T857 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 241782 T857 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 241784 T857 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 241785 T857 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 241787 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 241787 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 241790 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 241791 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 241794 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 241794 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 241801 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 241801 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 241804 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 241804 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 241806 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 241806 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 241808 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 241808 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 241811 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 241811 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 241813 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 241813 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 241815 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 241816 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 241818 T857 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 241818 T857 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 242063 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 242066 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58292
[junit4] 2> 242066 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 242066 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 242067 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509
[junit4] 2> 242067 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509/'
[junit4] 2> 242078 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509/solr.xml
[junit4] 2> 242133 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509
[junit4] 2> 242134 T857 oasc.CoreContainer.<init> New CoreContainer 26698920
[junit4] 2> 242134 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509/]
[junit4] 2> 242135 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 242136 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 242136 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 242136 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 242137 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 242137 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 242137 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 242137 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 242138 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 242138 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 242139 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 242139 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 242139 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 242140 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 242141 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 242144 T877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5164df name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 242144 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 242146 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 242147 T879 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c25810 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 242147 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 242149 T857 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 242151 T857 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 242153 T857 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 242156 T857 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 242157 T857 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 242159 T857 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 242161 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58292__fh
[junit4] 2> 242163 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58292__fh
[junit4] 2> 242165 T857 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 242168 T857 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 242171 T857 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:58292__fh
[junit4] 2> 242171 T857 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 242172 T857 oasc.Overseer.start Overseer (id=91471531752816643-127.0.0.1:58292__fh-n_0000000000) starting
[junit4] 2> 242174 T857 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 242181 T881 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 242182 T857 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 242182 T881 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 242184 T857 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 242185 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 242193 T880 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 242194 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509
[junit4] 2> 242200 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509/conf/
[junit4] 2> 242321 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 242322 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 242322 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 242323 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 242325 T884 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@117d777 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 242325 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 242327 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 242328 T857 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 242587 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 242587 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 242589 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:54225
[junit4] 2> 242590 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 242590 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 242591 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017
[junit4] 2> 242591 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/'
[junit4] 2> 242603 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/solr.xml
[junit4] 2> 242652 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017
[junit4] 2> 242652 T857 oasc.CoreContainer.<init> New CoreContainer 27725671
[junit4] 2> 242653 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/]
[junit4] 2> 242654 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 242654 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 242654 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 242655 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 242655 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 242655 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 242656 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 242656 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 242656 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 242657 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 242657 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 242657 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 242658 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 242658 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 242659 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 242660 T895 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e45b05 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 242660 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 242663 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 242664 T897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4dc395 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 242664 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 242668 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 243671 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54225__fh
[junit4] 2> 243672 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54225__fh
[junit4] 2> 243674 T879 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 243674 T884 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 243674 T897 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 243680 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017
[junit4] 2> 243686 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/conf/
[junit4] 2> 243809 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 243810 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 243810 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 244042 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 244042 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 244044 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44816
[junit4] 2> 244045 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 244045 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 244046 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499
[junit4] 2> 244046 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499/'
[junit4] 2> 244059 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499/solr.xml
[junit4] 2> 244118 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499
[junit4] 2> 244119 T857 oasc.CoreContainer.<init> New CoreContainer 13054120
[junit4] 2> 244119 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499/]
[junit4] 2> 244121 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 244121 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 244122 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 244122 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 244123 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 244123 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 244123 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 244124 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 244124 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 244125 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 244125 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 244126 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 244126 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 244127 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 244128 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 244129 T909 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14b80aa name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 244130 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 244131 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 244133 T911 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5476b6 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 244133 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 244138 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 245140 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44816__fh
[junit4] 2> 245142 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44816__fh
[junit4] 2> 245147 T884 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 245147 T879 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 245147 T911 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 245147 T897 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 245155 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499
[junit4] 2> 245165 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499/conf/
[junit4] 2> 245283 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 245284 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 245284 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 245529 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 245529 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 245531 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50199
[junit4] 2> 245532 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 245532 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 245532 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973
[junit4] 2> 245533 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973/'
[junit4] 2> 245544 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973/solr.xml
[junit4] 2> 245594 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973
[junit4] 2> 245594 T857 oasc.CoreContainer.<init> New CoreContainer 20556877
[junit4] 2> 245595 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973/]
[junit4] 2> 245596 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 245596 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 245596 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 245597 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 245597 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 245597 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 245598 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 245598 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 245598 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 245599 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 245599 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 245599 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 245600 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 245600 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 245600 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 245602 T923 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f1ce4 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 245602 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 245604 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 245605 T925 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d490 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 245606 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 245611 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 246613 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50199__fh
[junit4] 2> 246614 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50199__fh
[junit4] 2> 246616 T911 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 246617 T879 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 246617 T884 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 246617 T897 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 246617 T925 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 246622 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973
[junit4] 2> 246628 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973/conf/
[junit4] 2> 246750 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 246751 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 246751 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 246994 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 246995 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 246997 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43737
[junit4] 2> 246997 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 246998 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 246998 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440
[junit4] 2> 246999 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440/'
[junit4] 2> 247017 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440/solr.xml
[junit4] 2> 247083 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440
[junit4] 2> 247083 T857 oasc.CoreContainer.<init> New CoreContainer 2120319
[junit4] 2> 247084 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440/]
[junit4] 2> 247085 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 247085 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 247085 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 247086 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 247086 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 247086 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 247086 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 247087 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 247087 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 247088 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 247088 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 247088 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 247088 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 247089 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 247089 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 247091 T937 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1548f0 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 247091 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 247093 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 247094 T939 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a15ae5 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 247094 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 247099 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 248101 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43737__fh
[junit4] 2> 248102 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43737__fh
[junit4] 2> 248106 T911 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248106 T884 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248106 T925 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248106 T897 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248106 T879 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248106 T939 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 248110 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440
[junit4] 2> 248115 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440/conf/
[junit4] 2> 248211 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 248212 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 248213 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 248451 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 5
[junit4] 2> 248452 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 248454 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60168
[junit4] 2> 248454 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 248455 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 248455 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901
[junit4] 2> 248455 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901/'
[junit4] 2> 248472 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901/solr.xml
[junit4] 2> 248520 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901
[junit4] 2> 248521 T857 oasc.CoreContainer.<init> New CoreContainer 28259709
[junit4] 2> 248521 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901/]
[junit4] 2> 248522 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 248523 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 248523 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 248523 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 248523 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 248524 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 248524 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 248524 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 248525 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 248525 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 248525 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 248526 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 248526 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 248526 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 248527 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 248528 T951 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e0b0c4 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 248529 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 248532 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 248534 T953 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@131dfd9 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 248534 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 248539 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 249541 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60168__fh
[junit4] 2> 249542 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60168__fh
[junit4] 2> 249545 T879 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249545 T953 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249545 T911 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249545 T939 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249545 T884 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249545 T897 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249546 T925 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 249550 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901
[junit4] 2> 249555 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901/conf/
[junit4] 2> 249650 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 249651 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 249651 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 249903 T857 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 6
[junit4] 2> 249904 T857 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 249906 T857 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42431
[junit4] 2> 249906 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 249907 T857 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 249907 T857 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340
[junit4] 2> 249907 T857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/'
[junit4] 2> 249918 T857 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/solr.xml
[junit4] 2> 249965 T857 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340
[junit4] 2> 249965 T857 oasc.CoreContainer.<init> New CoreContainer 26197740
[junit4] 2> 249965 T857 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/]
[junit4] 2> 249967 T857 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 249967 T857 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 249967 T857 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 249967 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 249968 T857 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 249968 T857 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 249968 T857 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 249968 T857 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 249969 T857 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 249969 T857 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 249969 T857 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 249970 T857 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 249970 T857 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43489/solr
[junit4] 2> 249970 T857 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 249971 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 249972 T965 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f78c1 name:ZooKeeperConnection Watcher:127.0.0.1:43489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 249973 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 249975 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 249976 T967 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9956cb name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 249976 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 249981 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 250983 T857 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42431__fh
[junit4] 2> 250984 T857 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42431__fh
[junit4] 2> 250986 T911 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T879 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T939 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T967 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T897 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T953 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T925 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250987 T884 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 250992 T857 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340
[junit4] 2> 251001 T857 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/conf/
[junit4] 2> 251157 T857 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 251159 T857 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 251159 T857 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 251161 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 251162 T970 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11d40bc name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 251162 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 251163 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 251166 T857 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 251167 T972 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b36d74 name:ZooKeeperConnection Watcher:127.0.0.1:43489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 251167 T857 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 251168 T857 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 251173 T893 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&name=testOverseerCol&action=CREATE&version=2&wt=javabin&numShards=4&maxShardsPerNode=2
[junit4] 2> 251175 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251176 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251176 T881 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerCol",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 251176 T881 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerCol",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 251177 T881 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
[junit4] 2> 251178 T881 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol
[junit4] 2> 251178 T881 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
[junit4] 2> 251180 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251181 T880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 251181 T880 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
[junit4] 2> 251182 T880 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
[junit4] 2> 251183 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251184 T970 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T939 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T953 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T884 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T925 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T911 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251184 T967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251280 T881 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection, shardNames [shard1, shard2, shard3, shard4] , replicationFactor : 2
[junit4] 2> 251280 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:42431__fh
[junit4] 2> 251281 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:50199__fh
[junit4] 2> 251282 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:54225__fh
[junit4] 2> 251283 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:58292__fh
[junit4] 2> 251283 T963 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard1_replica1&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard1&numShards=4
[junit4] 2> 251284 T921 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard1_replica2&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard1&numShards=4
[junit4] 2> 251285 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:43737__fh
[junit4] 2> 251285 T890 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard2_replica1&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard2&numShards=4
[junit4] 2> 251286 T873 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard2_replica2&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard2&numShards=4
[junit4] 2> 251286 T963 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down collection=testOverseerCol
[junit4] 2> 251287 T890 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down collection=testOverseerCol
[junit4] 2> 251286 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:44816__fh
[junit4] 2> 251288 T933 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard3_replica1&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard3&numShards=4
[junit4] 2> 251288 T921 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down collection=testOverseerCol
[junit4] 2> 251289 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251289 T963 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251290 T921 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251289 T873 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down collection=testOverseerCol
[junit4] 2> 251289 T890 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251290 T880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 251290 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:60168__fh
[junit4] 2> 251290 T933 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down collection=testOverseerCol
[junit4] 2> 251292 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42431/_fh",
[junit4] 2> "core":"testOverseerCol_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42431__fh",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251291 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251291 T873 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251291 T904 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard3_replica2&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard3&numShards=4
[junit4] 2> 251294 T949 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard4_replica1&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard4&numShards=4
[junit4] 2> 251293 T933 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251292 T881 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:42431__fh
[junit4] 2> 251295 T904 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down collection=testOverseerCol
[junit4] 2> 251295 T949 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down collection=testOverseerCol
[junit4] 2> 251296 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54225/_fh",
[junit4] 2> "core":"testOverseerCol_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54225__fh",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251296 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251296 T904 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251297 T949 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251297 T962 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&name=testOverseerCol_shard4_replica2&collection.configName=conf1&action=CREATE&version=2&wt=javabin&collection=testOverseerCol&shard=shard4&numShards=4
[junit4] 2> 251299 T962 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down collection=testOverseerCol
[junit4] 2> 251300 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251300 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:50199/_fh",
[junit4] 2> "core":"testOverseerCol_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50199__fh",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251300 T962 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 251302 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58292/_fh",
[junit4] 2> "core":"testOverseerCol_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58292__fh",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251311 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251312 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43737/_fh",
[junit4] 2> "core":"testOverseerCol_shard3_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43737__fh",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251313 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251314 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44816/_fh",
[junit4] 2> "core":"testOverseerCol_shard3_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44816__fh",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251316 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251316 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60168/_fh",
[junit4] 2> "core":"testOverseerCol_shard4_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60168__fh",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251318 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251318 T880 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42431/_fh",
[junit4] 2> "core":"testOverseerCol_shard4_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42431__fh",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 251320 T879 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 251423 T970 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T939 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251424 T925 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251424 T884 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T953 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 251423 T911 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 252290 T963 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
[junit4] 2> 252290 T921 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
[junit4] 2> 252291 T890 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
[junit4] 2> 252291 T921 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252291 T963 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252292 T890 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252292 T921 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252292 T963 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252292 T890 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252293 T890 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252292 T921 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252294 T873 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
[junit4] 2> 252294 T890 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252295 T921 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252293 T963 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252295 T921 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395744811973/testOverseerCol_shard1_replica2/'
[junit4] 2> 252295 T873 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252295 T890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/'
[junit4] 2> 252297 T873 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252297 T873 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252295 T933 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
[junit4] 2> 252297 T904 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
[junit4] 2> 252296 T963 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252298 T933 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252298 T873 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252299 T904 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252298 T963 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard1_replica1/'
[junit4] 2> 252300 T949 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
[junit4] 2> 252300 T873 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395744808509/testOverseerCol_shard2_replica2/'
[junit4] 2> 252301 T904 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252302 T904 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252301 T962 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
[junit4] 2> 252302 T933 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252303 T933 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252301 T949 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252304 T904 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252305 T904 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395744810499/testOverseerCol_shard3_replica2/'
[junit4] 2> 252303 T962 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 252308 T933 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252308 T962 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252308 T933 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1395744813440/testOverseerCol_shard3_replica1/'
[junit4] 2> 252309 T949 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 252309 T949 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252309 T962 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 252311 T949 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252312 T949 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty5-1395744814901/testOverseerCol_shard4_replica1/'
[junit4] 2> 252312 T962 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 252313 T962 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/'
[junit4] 2> 252387 T921 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252413 T890 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252415 T933 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252471 T904 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252472 T873 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252491 T963 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252509 T949 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252512 T962 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 252528 T921 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252534 T921 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252555 T921 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
[junit4] 2> 252590 T933 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252596 T933 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252597 T890 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252598 T890 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252614 T873 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252614 T933 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
[junit4] 2> 252619 T890 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
[junit4] 2> 252626 T963 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252627 T963 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252629 T873 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252642 T873 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
[junit4] 2> 252653 T963 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
[junit4] 2> 252656 T962 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252657 T962 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252675 T962 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
[junit4] 2> 252676 T904 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252677 T949 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 252679 T904 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252679 T949 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 252698 T904 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
[junit4] 2> 252701 T949 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
[junit4] 2> 253882 T890 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 253922 T890 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 253927 T890 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 253937 T962 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 253953 T962 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 253959 T962 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 253961 T949 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 253965 T963 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 253966 T890 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 253980 T890 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 253981 T963 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 253982 T962 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 253982 T949 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 253982 T921 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 253986 T962 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 253987 T963 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 253988 T949 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 253986 T890 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 253993 T890 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 253995 T890 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 253995 T921 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 253996 T890 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 254003 T890 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 254003 T890 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 254003 T962 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 254004 T890 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica1' using configuration from collection testOverseerCol
[junit4] 2> 254004 T890 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 254005 T890 oasc.SolrCore.<init> [testOverseerCol_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/data/
[junit4] 2> 254005 T890 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fd193
[junit4] 2> 254006 T890 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/data
[junit4] 2> 254007 T890 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/data/index/
[junit4] 2> 254007 T921 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 254007 T890 oasc.SolrCore.initIndex WARN [testOverseerCol_shard2_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 254008 T890 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395744809017/testOverseerCol_shard2_replica1/data/index
[junit4] 2> 254009 T890 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 254010 T963 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 254010 T962 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 254011 T962 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 254011 T962 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 254010 T890 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@a24b31 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1447bdc),segFN=segments_1,generation=1}
[junit4] 2> 254012 T890 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 254013 T962 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 254013 T962 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 254014 T962 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica2' using configuration from collection testOverseerCol
[junit4] 2> 254014 T962 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 254014 T962 oasc.SolrCore.<init> [testOverseerCol_shard4_replica2] Opening new SolrCore at ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/, dataDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/data/
[junit4] 2> 254015 T962 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fd193
[junit4] 2> 254015 T963 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 254021 T962 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/data
[junit4] 2> 254021 T963 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 254022 T962 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/data/index/
[junit4] 2> 254023 T962 oasc.SolrCore.initIndex WARN [testOverseerCol_shard4_replica2] Solr index directory './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395744816340/testOverseerCol_shard4_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 254024 T963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 254024 T963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 254022 T890 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 254025 T963 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4]
[...truncated too long message...]
base_url":"http://127.0.0.1:42431/_fh",
[junit4] 2> "core":"testOverseerCol_shard4_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42431__fh",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":"core_node8"}
[junit4] 2> 265969 T1032 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 265982 T857 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 265983 T857 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 265983 T857 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 265983 T857 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 265984 T857 oasc.SolrCore.closeSearcher [testOverseerCol_shard1_replica1] Closing main searcher on request.
[junit4] 2> 265985 T857 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 265985 T857 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6;done=false>>]
[junit4] 2> 265985 T857 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6
[junit4] 2> 265986 T857 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index;done=false>>]
[junit4] 2> 265986 T857 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index
[junit4] 2> 265986 T857 oasc.SolrCore.close [testOverseerCol_shard4_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@1f137a7
[junit4] 2> 266010 T857 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 266010 T857 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 266011 T857 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 266011 T857 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 266012 T857 oasc.SolrCore.closeSearcher [testOverseerCol_shard4_replica2] Closing main searcher on request.
[junit4] 2> 266012 T857 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 266012 T857 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6;done=false>>]
[junit4] 2> 266013 T857 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6
[junit4] 2> 266013 T857 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index;done=false>>]
[junit4] 2> 266013 T857 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-1395744808238/jetty6/index
[junit4] 2> 266016 T884 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 266016 T970 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 266037 T857 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_fh,null}
[junit4] 2> 266091 T857 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 58292
[junit4] 2> 266093 T857 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 266095 T857 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:43489 43489
[junit4] 2> 266396 T939 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 267515 T1032 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 266488 T953 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerRolesTest -Dtests.method=testDistribSearch -Dtests.seed=F17FBEC930E6049 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_IQ -Dtests.timezone=Asia/Riyadh87 -Dtests.file.encoding=UTF-8
[junit4] ERROR 34.6s J0 | OverseerRolesTest.testDistribSearch <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1034, name=Overseer-91471531752816648-127.0.0.1:44816__fh-n_0000000010, state=RUNNABLE, group=Overseer collection creation process.]
[junit4] > at __randomizedtesting.SeedInfo.seed([F17FBEC930E6049:8EF175F4E4510075]:0)
[junit4] > Caused by: org.apache.solr.common.cloud.ZooKeeperException:
[junit4] > at __randomizedtesting.SeedInfo.seed([F17FBEC930E6049]:0)
[junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:274)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000006
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
[junit4] > at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:206)
[junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:257)
[junit4] > ... 1 more
[junit4] 2> 276175 T857 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_IQ, timezone=Asia/Riyadh87
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_51 (32-bit)/cpus=8,threads=1,free=27105752,total=96645120
[junit4] 2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, TestManagedResource, LukeRequestHandlerTest, TestWriterPerf, UnloadDistributedZkTest, SOLR749Test, SynonymTokenizerTest, SimpleFacetsTest, DistributedExpandComponentTest, DistributedQueryElevationComponentTest, TestStressVersions, TestFreeTextSuggestions, PeerSyncTest, SliceStateUpdateTest, TestUniqueKeyFieldResource, TestRemoteStreaming, AutoCommitTest, OverseerCollectionProcessorTest, DeleteShardTest, TestSolrXmlPersistor, CopyFieldTest, TestRealTimeGet, MBeansHandlerTest, PolyFieldTest, TestInitQParser, FileBasedSpellCheckerTest, TestLRUCache, TestExtendedDismaxParser, SearchHandlerTest, DistributedQueryComponentCustomSortTest, CoreMergeIndexesAdminHandlerTest, TestSolrXMLSerializer, TestImplicitCoreProperties, TestManagedResourceStorage, TestPerFieldSimilarity, OverseerRolesTest]
[junit4] Completed on J0 in 34.66s, 1 test, 1 error <<< FAILURES!
[...truncated 1027 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 393 suites, 1650 tests, 1 error, 49 ignored (28 assumptions)
Total time: 60 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b10) -
Build # 9912 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9912/
Java: 64bit/jdk1.7.0_60-ea-b10 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58118 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58118 within 45000 ms
at __randomizedtesting.SeedInfo.seed([6E16AA3B66C6C381:EFF024231199A3BD]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58118 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11605 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 28328 T84 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 28328 T84 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 28330 T84 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 28334 T84 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395856000772
[junit4] 2> 28336 T84 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 28338 T85 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 28438 T84 oasc.ZkTestServer.run start zk server on port:58118
[junit4] 2> 28468 T84 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 93709 T88 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 65221ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 93710 T84 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 93727 T86 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144ff8047320000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 93728 T86 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 93729 T86 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 93730 T86 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 93731 T84 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58118 58118
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=6E16AA3B66C6C381 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_BO -Dtests.timezone=Atlantic/Faroe -Dtests.file.encoding=UTF-8
[junit4] ERROR 65.5s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58118 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([6E16AA3B66C6C381:EFF024231199A3BD]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58118 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 93842 T84 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395856066279
[junit4] 2> 93843 T84 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 93843 T92 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 93943 T84 oasc.ZkTestServer.run start zk server on port:53758
[junit4] 2> 93944 T84 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 93956 T98 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@182d4c7f name:ZooKeeperConnection Watcher:127.0.0.1:53758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 93956 T84 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 93959 T84 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 93981 T84 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 93982 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b9389ee name:ZooKeeperConnection Watcher:127.0.0.1:53758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 93983 T84 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 93986 T84 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 93991 T84 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 93994 T84 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 94002 T84 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 94006 T84 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 94007 T84 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 94017 T84 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 94018 T84 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 94034 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 94035 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 94035 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 94035 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 94036 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 94036 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 94037 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 94037 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 94038 T84 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 94049 T84 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 94050 T101 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 94050 T101 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:465)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:457)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 95152 T101 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 95254 T84 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 95256 T84 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53758 53758
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 95422 T84 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 67101 T83 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=285), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=285)), sim=DefaultSimilarity, locale=es_BO, timezone=Atlantic/Faroe
[junit4] 2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=119970504,total=219152384
[junit4] 2> NOTE: All tests run in this JVM: [SolrExampleStreamingBinaryTest, ModifiableSolrParamsTest, DocumentAnalysisResponseTest, ClientUtilsTest, SolrPingTest, TestDocumentObjectBinder, LargeVolumeJettyTest, AnlysisResponseBaseTest, CloudSolrServerTest]
[junit4] Completed on J0 in 67.38s, 2 tests, 1 error <<< FAILURES!
[...truncated 97 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 52 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b10 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b10) -
Build # 9911 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9911/
Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([CDB667E8A1E7CCF3]:0)
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([CDB667E8A1E7CCF3]:0)
Build Log:
[...truncated 11371 lines...]
[junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4] 2> 739012 T4588 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 739012 T4588 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /qdw/
[junit4] 2> 739016 T4588 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1395831395260
[junit4] 2> 739016 T4588 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 739017 T4589 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 739117 T4588 oasc.ZkTestServer.run start zk server on port:59894
[junit4] 2> 739118 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 739128 T4595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c38f7ad name:ZooKeeperConnection Watcher:127.0.0.1:59894 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 739128 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 739128 T4588 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 739132 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 739133 T4597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73a41d1b name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 739134 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 739134 T4588 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 739136 T4588 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 739137 T4588 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 739138 T4588 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 739139 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 739140 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 739142 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 739142 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 739144 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 739145 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 739146 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 739146 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 739149 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 739149 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 739150 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 739151 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 739152 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 739152 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 739154 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 739155 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 739156 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 739156 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 739158 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 739158 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 739160 T4588 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 739161 T4588 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 739163 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 739164 T4599 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7733ee56 name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 739164 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 739374 T4588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 739377 T4588 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 739380 T4588 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:60275
[junit4] 2> 739382 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 739382 T4588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 739382 T4588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409
[junit4] 2> 739383 T4588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/'
[junit4] 2> 739395 T4588 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/solr.xml
[junit4] 2> 739426 T4588 oasc.CoreContainer.<init> New CoreContainer 1055075769
[junit4] 2> 739427 T4588 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/]
[junit4] 2> 739427 T4588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 739428 T4588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 739428 T4588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 739428 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 739428 T4588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 739429 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 739429 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 739429 T4588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 739429 T4588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 739431 T4588 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 739431 T4588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 739432 T4588 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 739432 T4588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59894/solr
[junit4] 2> 739432 T4588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 739433 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 739434 T4610 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5af6945c name:ZooKeeperConnection Watcher:127.0.0.1:59894 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 739434 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 739436 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 739437 T4612 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47eadf35 name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 739437 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 739438 T4588 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 739440 T4588 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 739441 T4588 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 739443 T4588 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 739445 T4588 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 739446 T4588 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 739448 T4588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60275_qdw
[junit4] 2> 739450 T4588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60275_qdw
[junit4] 2> 739451 T4588 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 739453 T4588 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 739460 T4588 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:60275_qdw
[junit4] 2> 739460 T4588 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 739462 T4588 oasc.Overseer.start Overseer (id=91477206319890436-127.0.0.1:60275_qdw-n_0000000000) starting
[junit4] 2> 739464 T4588 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 739471 T4614 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 739471 T4588 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 739471 T4614 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 739472 T4588 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 739473 T4588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 739475 T4613 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 739478 T4615 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 739478 T4615 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 739479 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 739479 T4615 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 739480 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 739481 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:60275/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60275_qdw",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 739481 T4613 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 739481 T4613 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 739485 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 739485 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 740480 T4615 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 740480 T4615 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 740481 T4615 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 740481 T4615 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 740482 T4615 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 740482 T4615 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/collection1/'
[junit4] 2> 740482 T4615 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/collection1/lib/.svn/' to classloader
[junit4] 2> 740483 T4615 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/collection1/lib/classes/' to classloader
[junit4] 2> 740483 T4615 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/collection1/lib/README' to classloader
[junit4] 2> 740508 T4615 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 740539 T4615 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 740541 T4615 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 740546 T4615 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 740816 T4615 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 740817 T4615 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 740818 T4615 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 740821 T4615 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 740836 T4615 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 740836 T4615 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 740836 T4615 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1395831395409/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/control/data/
[junit4] 2> 740836 T4615 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41d1465c
[junit4] 2> 740837 T4615 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/control/data
[junit4] 2> 740837 T4615 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/control/data/index/
[junit4] 2> 740837 T4615 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1395831395260/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 740837 T4615 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/control/data/index
[junit4] 2> 740838 T4615 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=23, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.314084743067576]
[junit4] 2> 740838 T4615 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3e63d565 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@105725a4),segFN=segments_1,generation=1}
[junit4] 2> 740838 T4615 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 740842 T4615 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 740842 T4615 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 740842 T4615 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 740842 T4615 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 740842 T4615 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 740843 T4615 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 740843 T4615 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 740843 T4615 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 740843 T4615 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 740844 T4615 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 740844 T4615 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 740844 T4615 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 740844 T4615 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 740845 T4615 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 740845 T4615 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 740845 T4615 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 740846 T4615 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 740848 T4615 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 740849 T4615 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 740850 T4615 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 740850 T4615 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=74.349609375, floorSegmentMB=1.142578125, forceMergeDeletesPctAllowed=5.120953378894949, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18522701469733677
[junit4] 2> 740850 T4615 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3e63d565 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@105725a4),segFN=segments_1,generation=1}
[junit4] 2> 740851 T4615 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 740851 T4615 oass.SolrIndexSearcher.<init> Opening Searcher@4ada27b0[collection1] main
[junit4] 2> 740851 T4615 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 740852 T4615 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 740852 T4615 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 740852 T4615 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 740852 T4615 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 740852 T4615 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 740853 T4615 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 740853 T4615 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 740853 T4615 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 740853 T4615 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 740855 T4616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ada27b0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 740856 T4615 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 740865 T4619 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:60275/qdw collection:control_collection shard:shard1
[junit4] 2> 740865 T4588 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 740866 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 740866 T4619 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 740869 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 740870 T4619 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 740870 T4621 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c85b274 name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 740871 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 740871 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 740871 T4588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 740872 T4619 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 740872 T4619 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2104 name=collection1 org.apache.solr.core.SolrCore@12a7a3f1 url=https://127.0.0.1:60275/qdw/collection1 node=127.0.0.1:60275_qdw C2104_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:60275/qdw, core=collection1, node_name=127.0.0.1:60275_qdw}
[junit4] 2> 740872 T4619 C2104 P60275 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:60275/qdw/collection1/
[junit4] 2> 740872 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 740872 T4619 C2104 P60275 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 740873 T4619 C2104 P60275 oasc.SyncStrategy.syncToMe https://127.0.0.1:60275/qdw/collection1/ has no replicas
[junit4] 2> 740873 T4619 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:60275/qdw/collection1/ shard1
[junit4] 2> 740873 T4619 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 740873 T4588 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 740874 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 740878 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 740881 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 740982 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 740982 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 741029 T4619 oasc.ZkController.register We are https://127.0.0.1:60275/qdw/collection1/ and leader is https://127.0.0.1:60275/qdw/collection1/
[junit4] 2> 741029 T4619 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:60275/qdw
[junit4] 2> 741029 T4619 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 741029 T4619 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 741029 T4619 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 741030 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 741030 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 741030 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 741030 T4619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 741031 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 741032 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:60275/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60275_qdw",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 741033 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 741092 T4588 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 741092 T4588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 741095 T4588 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 741096 T4588 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:40052
[junit4] 2> 741098 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 741098 T4588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 741098 T4588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118
[junit4] 2> 741099 T4588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/'
[junit4] 2> 741111 T4588 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/solr.xml
[junit4] 2> 741137 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 741137 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 741146 T4588 oasc.CoreContainer.<init> New CoreContainer 58481025
[junit4] 2> 741146 T4588 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/]
[junit4] 2> 741147 T4588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 741147 T4588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 741148 T4588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 741148 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 741148 T4588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 741148 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 741148 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 741149 T4588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 741149 T4588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 741151 T4588 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 741151 T4588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 741151 T4588 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 741152 T4588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59894/solr
[junit4] 2> 741152 T4588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 741152 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 741155 T4632 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10ad14a8 name:ZooKeeperConnection Watcher:127.0.0.1:59894 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 741155 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 741156 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 741157 T4634 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@170e8785 name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 741158 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 741161 T4588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 742163 T4588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40052_qdw
[junit4] 2> 742164 T4588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40052_qdw
[junit4] 2> 742166 T4621 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 742166 T4634 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 742166 T4612 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 742171 T4635 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 742171 T4635 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 742172 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 742172 T4635 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 742172 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 742172 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 742173 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 742173 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40052/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40052_qdw",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 742173 T4613 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 742173 T4613 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 742179 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 742281 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 742281 T4634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 742281 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743172 T4635 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 743172 T4635 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 743173 T4635 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 743173 T4635 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 743174 T4635 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 743174 T4635 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/collection1/'
[junit4] 2> 743174 T4635 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/collection1/lib/.svn/' to classloader
[junit4] 2> 743175 T4635 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/collection1/lib/classes/' to classloader
[junit4] 2> 743175 T4635 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/collection1/lib/README' to classloader
[junit4] 2> 743212 T4635 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 743242 T4635 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 743243 T4635 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 743249 T4635 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 743497 T4635 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 743499 T4635 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 743500 T4635 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 743503 T4635 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 743527 T4635 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 743528 T4635 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 743528 T4635 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1395831397118/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty1/
[junit4] 2> 743528 T4635 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41d1465c
[junit4] 2> 743529 T4635 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty1
[junit4] 2> 743529 T4635 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty1/index/
[junit4] 2> 743529 T4635 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 743529 T4635 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty1/index
[junit4] 2> 743530 T4635 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=23, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.314084743067576]
[junit4] 2> 743530 T4635 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@683d8bb6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23e772d0),segFN=segments_1,generation=1}
[junit4] 2> 743530 T4635 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 743535 T4635 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 743535 T4635 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 743535 T4635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 743535 T4635 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 743536 T4635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 743536 T4635 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 743536 T4635 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 743537 T4635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 743537 T4635 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 743537 T4635 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 743538 T4635 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 743538 T4635 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 743538 T4635 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 743538 T4635 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 743539 T4635 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 743539 T4635 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 743540 T4635 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 743542 T4635 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 743544 T4635 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 743544 T4635 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 743544 T4635 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=74.349609375, floorSegmentMB=1.142578125, forceMergeDeletesPctAllowed=5.120953378894949, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18522701469733677
[junit4] 2> 743545 T4635 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@683d8bb6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23e772d0),segFN=segments_1,generation=1}
[junit4] 2> 743545 T4635 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 743545 T4635 oass.SolrIndexSearcher.<init> Opening Searcher@114c0ac[collection1] main
[junit4] 2> 743545 T4635 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 743546 T4635 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 743546 T4635 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 743547 T4635 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 743547 T4635 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 743547 T4635 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 743547 T4635 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 743548 T4635 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 743548 T4635 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 743548 T4635 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 743550 T4636 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@114c0ac[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 743552 T4635 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 743553 T4639 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:40052/qdw collection:collection1 shard:shard1
[junit4] 2> 743553 T4588 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 743553 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 743556 T4639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 743561 T4639 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 743562 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743562 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743563 T4639 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 743563 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743563 T4639 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2105 name=collection1 org.apache.solr.core.SolrCore@2863cd79 url=https://127.0.0.1:40052/qdw/collection1 node=127.0.0.1:40052_qdw C2105_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:40052/qdw, core=collection1, node_name=127.0.0.1:40052_qdw}
[junit4] 2> 743563 T4639 C2105 P40052 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:40052/qdw/collection1/
[junit4] 2> 743563 T4639 C2105 P40052 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 743563 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 743563 T4639 C2105 P40052 oasc.SyncStrategy.syncToMe https://127.0.0.1:40052/qdw/collection1/ has no replicas
[junit4] 2> 743564 T4639 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:40052/qdw/collection1/ shard1
[junit4] 2> 743564 T4639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 743565 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743567 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743569 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743671 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743671 T4634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743671 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743717 T4639 oasc.ZkController.register We are https://127.0.0.1:40052/qdw/collection1/ and leader is https://127.0.0.1:40052/qdw/collection1/
[junit4] 2> 743718 T4639 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:40052/qdw
[junit4] 2> 743718 T4639 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 743718 T4639 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 743718 T4639 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 743719 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743719 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743719 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743719 T4639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 743720 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 743721 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:40052/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40052_qdw",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 743723 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 743768 T4588 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 743769 T4588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 743771 T4588 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 743772 T4588 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:58364
[junit4] 2> 743774 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 743775 T4588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 743775 T4588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799
[junit4] 2> 743775 T4588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/'
[junit4] 2> 743787 T4588 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/solr.xml
[junit4] 2> 743820 T4588 oasc.CoreContainer.<init> New CoreContainer 864195439
[junit4] 2> 743820 T4588 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/]
[junit4] 2> 743821 T4588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 743821 T4588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 743821 T4588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 743821 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 743821 T4588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 743822 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 743822 T4588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 743822 T4588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 743822 T4588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 743824 T4634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743824 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743824 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 743824 T4588 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 743825 T4588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 743825 T4588 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 743826 T4588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59894/solr
[junit4] 2> 743826 T4588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 743827 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 743828 T4650 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13710722 name:ZooKeeperConnection Watcher:127.0.0.1:59894 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 743828 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 743829 T4588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 743830 T4652 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@155efc72 name:ZooKeeperConnection Watcher:127.0.0.1:59894/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 743830 T4588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 743835 T4588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 744836 T4588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58364_qdw
[junit4] 2> 744838 T4588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58364_qdw
[junit4] 2> 744839 T4634 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 744839 T4652 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 744839 T4612 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 744839 T4621 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 744845 T4653 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 744845 T4653 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 744846 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 744846 T4653 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 744846 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 744847 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 744848 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 744849 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58364/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58364_qdw",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 744849 T4613 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 744849 T4613 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 744856 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 744957 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 744957 T4634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 744957 T4652 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 744957 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 745847 T4653 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 745847 T4653 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 745848 T4653 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 745848 T4653 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 745849 T4653 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 745849 T4653 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/collection1/'
[junit4] 2> 745849 T4653 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/collection1/lib/.svn/' to classloader
[junit4] 2> 745849 T4653 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/collection1/lib/classes/' to classloader
[junit4] 2> 745850 T4653 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/collection1/lib/README' to classloader
[junit4] 2> 745900 T4653 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 745939 T4653 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 745940 T4653 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 745946 T4653 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 746210 T4653 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 746212 T4653 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 746213 T4653 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 746216 T4653 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 746233 T4653 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 746233 T4653 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 746233 T4653 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.RecoveryZkTest-jetty2-1395831399799/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty2/
[junit4] 2> 746233 T4653 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41d1465c
[junit4] 2> 746234 T4653 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty2
[junit4] 2> 746234 T4653 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty2/index/
[junit4] 2> 746235 T4653 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 746235 T4653 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1395831395260/jetty2/index
[junit4] 2> 746235 T4653 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=23, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.314084743067576]
[junit4] 2> 746236 T4653 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@34dd709f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@60fa3b38),segFN=segments_1,generation=1}
[junit4] 2> 746237 T4653 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 746243 T4653 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 746243 T4653 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 746244 T4653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 746244 T4653 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 746244 T4653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 746244 T4653 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 746245 T4653 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 746245 T4653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 746245 T4653 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 746246 T4653 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 746247 T4653 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 746247 T4653 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 746247 T4653 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 746248 T4653 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 746248 T4653 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 746249 T4653 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 746249 T4653 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 746252 T4653 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 746254 T4653 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 746254 T4653 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 746255 T4653 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=74.349609375, floorSegmentMB=1.142578125, forceMergeDeletesPctAllowed=5.120953378894949, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18522701469733677
[junit4] 2> 746255 T4653 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@34dd709f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@60fa3b38),segFN=segments_1,generation=1}
[junit4] 2> 746255 T4653 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 746256 T4653 oass.SolrIndexSearcher.<init> Opening Searcher@72059540[collection1] main
[junit4] 2> 746256 T4653 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 746257 T4653 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 746257 T4653 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 746257 T4653 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 746257 T4653 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 746257 T4653 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 746258 T4653 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 746258 T4653 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 746258 T4653 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 746258 T4653 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 746261 T4654 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72059540[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 746263 T4653 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 746264 T4657 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:58364/qdw collection:collection1 shard:shard1
[junit4] 2> 746264 T4588 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 746264 T4588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 746266 T4588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 746267 T4657 oasc.ZkController.register We are https://127.0.0.1:58364/qdw/collection1/ and leader is https://127.0.0.1:40052/qdw/collection1/
[junit4] 2> 746267 T4657 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:58364/qdw
[junit4] 2> 746268 T4657 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 746268 T4657 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C2106 name=collection1 org.apache.solr.core.SolrCore@5d5ec64 url=https://127.0.0.1:58364/qdw/collection1 node=127.0.0.1:58364_qdw C2106_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:58364/qdw, core=collection1, node_name=127.0.0.1:58364_qdw}
[junit4] 2> 746268 T4659 C2106 P58364 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 746268 T4657 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 746270 T4659 C2106 P58364 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 746270 T4659 C2106 P58364 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 746271 T4659 C2106 P58364 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 746272 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746272 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746272 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746273 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 746273 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"https://127.0.0.1:58364/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58364_qdw",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 746277 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746330 T4628 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 746331 T4628 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 746332 T4628 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> ASYNC NEW_CORE C2107 name=collection1 org.apache.solr.core.SolrCore@12a7a3f1 url=https://127.0.0.1:60275/qdw/collection1 node=127.0.0.1:60275_qdw C2107_STATE=coll:control_collection core:collection1 props:{state=active, base_url=https://127.0.0.1:60275/qdw, core=collection1, node_name=127.0.0.1:60275_qdw, leader=true}
[junit4] 2> 746340 T4608 C2107 P60275 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin&CONTROL=TRUE} {add=[2-0 (1463635308793823232)]} 0 2
[junit4] 2> 746344 T4607 C2107 P60275 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin&CONTROL=TRUE} {add=[1-0 (1463635308799066112)]} 0 1
[junit4] 2> 746381 T4621 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 746381 T4652 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 746381 T4634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 746381 T4612 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> C2106_STATE=coll:collection1 core:collection1 props:{state=recovering, base_url=https://127.0.0.1:58364/qdw, core=collection1, node_name=127.0.0.1:58364_qdw}
[junit4] 2> 746460 T4647 C2106 P58364 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={distrib.from=https://127.0.0.1:40052/qdw/collection1/&version=2&update.distrib=FROMLEADER&wt=javabin} {add=[2-0 (1463635308856737792)]} 0 2
[junit4] 2> ASYNC NEW_CORE C2108 name=collection1 org.apache.solr.core.SolrCore@2863cd79 url=https://127.0.0.1:40052/qdw/collection1 node=127.0.0.1:40052_qdw C2108_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:40052/qdw, core=collection1, node_name=127.0.0.1:40052_qdw, leader=true}
[junit4] 2> 746460 T4630 C2108 P40052 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin} {add=[2-0 (1463635308856737792)]} 0 62
[junit4] 2> 746467 T4645 C2106 P58364 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={distrib.from=https://127.0.0.1:40052/qdw/collection1/&version=2&update.distrib=FROMLEADER&wt=javabin} {add=[1-0 (1463635308866174976)]} 0 1
[junit4] 2> 746468 T4627 C2108 P40052 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin} {add=[1-0 (1463635308866174976)]} 0 61
[junit4] 2> 746468 T4588 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 58364
[junit4] 2> 746469 T4588 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=864195439
[junit4] 2> 746470 T4588 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=core_node2core=collection1
[junit4] 2> 746471 T4659 C2106 P58364 oasc.SolrException.log ERROR Error while trying to recover. core=collection1:java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:40052/qdw
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:611)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:370)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
[junit4] 2> Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:40052/qdw
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:565)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:244)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:240)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> Caused by: java.net.SocketException: Socket closed
[junit4] 2> at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:152)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:122)
[junit4] 2> at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
[junit4] 2> at sun.security.ssl.InputRecord.read(InputRecord.java:480)
[junit4] 2> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
[junit4] 2> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
[junit4] 2> at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
[junit4] 2> at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
[junit4] 2> at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
[junit4] 2> at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
[junit4] 2> at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
[junit4] 2> at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
[junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
[junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
[junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
[junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:451)
[junit4] 2> ... 6 more
[junit4] 2>
[junit4] 2> 746471 T4659 C2106 P58364 oasc.RecoveryStrategy.doRecovery ERROR Recovery failed - trying again... (0) core=collection1
[junit4] 2> 746472 T4659 C2106 P58364 oasc.SolrException.log ERROR Recovery failed - interrupted. core=collection1
[junit4] 2> 746472 T4659 C2106 P58364 oasc.SolrException.log ERROR Recovery failed - I give up. core=collection1
[junit4] 2> 746472 T4659 C2106 P58364 oasc.ZkController.publish publishing core=collection1 state=recovery_failed collection=collection1
[junit4] 2> 746472 T4659 C2106 P58364 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 746472 T4607 C2107 P60275 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin&CONTROL=TRUE} {add=[2-1 (1463635308934332416)]} 0 0
[junit4] 2> 746473 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746474 T4659 C2106 P58364 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=core_node2core=collection1
[junit4] 2> 746474 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 746475 T4659 C2106 P58364 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4] 2> 746474 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746475 T4588 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 746475 T4588 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 746475 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746476 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovery_failed",
[junit4] 2> "base_url":"https://127.0.0.1:58364/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58364_qdw",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 746478 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746478 T4588 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5d5ec64
[junit4] 2> 746481 T4605 C2107 P60275 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin&CONTROL=TRUE} {add=[1-1 (1463635308942721024)]} 0 1
[junit4] 2> 746481 T4613 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58364/qdw",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58364_qdw",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 746484 T4612 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 746484 T4663 oasu.StreamingSolrServers$1.handleError ERROR error org.apache.solr.common.SolrException: Can not find: /qdw/collection1/update
[junit4] 2>
[junit4] 2>
[junit4] 2>
[junit4] 2> request: https://127.0.0.1:58364/qdw/collection1/update?update.distrib=FROMLEADER&distrib.from=https%3A%2F%2F127.0.0.1%3A40052%2Fqdw%2Fcollection1%2F&wt=javabin&version=2
[junit4] 2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> ASYNC NEW_CORE C2109 name=collection1 org.apache.solr.core.SolrCore@2863cd79 url=https://127.0.0.1:40052/qdw/collection1 node=127.0.0.1:40052_qdw C2109_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:40052/qdw, core=collection1, node_name=127.0.0.1:40052_qdw, leader=true}
[junit4] 2> 746495 T4630 C2109 P40052 oasup.DistributedUpdateProcessor.doFinish WARN Error sending update org.apache.solr.common.SolrException: Can not find: /qdw/collection1/update
[junit4] 2>
[junit4] 2>
[junit4] 2>
[junit4] 2> request: https://127.0.0.1:58364/qdw/collection1/update?update.distrib=FROMLEADER&distrib.from=https%3A%2F%2F127.0.0.1%3A40052%2Fqdw%2Fcollection1%2F&wt=javabin&version=2
[junit4] 2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 746496 T4663 oasup.DistributedUpdateProcessor$1.run try and ask https://127.0.0.1:58364/qdw to recover
[junit4] 2> 746496 T4630 C2109 P40052 oasup.LogUpdateProcessor.finish [collection1] webapp=/qdw path=/update params={version=2&wt=javabin} {add=[2-1 (1463635308939575296)]} 0 19
[junit4] 2> 746497 T4588 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=2,adds=2,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 746497 T4588 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 746497 T4588 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 746497 T4588 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> ASYNC NEW_CORE C2110 name=collection1 org.apache.solr.core.SolrCore@5d5ec64 url=https://127.0.0.1:58364/qdw/collection1 node=127.0.0.1:58364_qdw C2110_STATE=coll:collection1 core:collection1 props:{state=recovering, base_url=https://127.0.0.1:58364/qdw, core=collection1, node_name=127.0.0.1:58364_qdw}
[junit4] 2> 746500 T4588 C2110 P58364 oasc
[...truncated too long message...]
QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 14) Thread[id=4680, name=searcherExecutor-2062-thread-1, state=WAITING, group=TGRP-RecoveryZkTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 15) Thread[id=4668, name=qtp522691791-4668 Selector1, state=RUNNABLE, group=TGRP-RecoveryZkTest]
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 16) Thread[id=4639, name=coreZkRegister-2042-thread-1, state=WAITING, group=TGRP-RecoveryZkTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 17) Thread[id=4633, name=TEST-RecoveryZkTest.testDistribSearch-seed#[CDB667E8A1E7CCF3]-SendThread(127.0.0.1:59894), state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> 18) Thread[id=4624, name=qtp1550174314-4624 Selector1, state=RUNNABLE, group=TGRP-RecoveryZkTest]
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 19) Thread[id=4669, name=qtp522691791-4669 Acceptor0 SslSelectChannelConnector@127.0.0.1:58364, state=BLOCKED, group=TGRP-RecoveryZkTest]
[junit4] 2> at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> NOTE: test params are: codec=Lucene46: {id=MockFixedIntBlock(blockSize=1830), a_si=MockVariableIntBlock(baseBlockSize=125), rnd_b=MockFixedIntBlock(blockSize=1830), _version_=PostingsFormat(name=FSTPulsing41), a_t=PostingsFormat(name=FSTPulsing41)}, docValues:{}, sim=DefaultSimilarity, locale=hr_HR, timezone=Pacific/Port_Moresby
[junit4] 2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=25,free=215069208,total=334983168
[junit4] 2> NOTE: All tests run in this JVM: [TestNoOpRegenerator, SolrInfoMBeanTest, TestFreeTextSuggestions, TestComplexPhraseQParserPlugin, StatelessScriptUpdateProcessorFactoryTest, SimpleFacetsTest, TestSolrXml, TestFileDictionaryLookup, TestDistribDocBasedVersion, CachingDirectoryFactoryTest, TestFieldTypeResource, UnloadDistributedZkTest, BasicDistributedZkTest, DistributedQueryComponentOptimizationTest, TestStressVersions, TermsComponentTest, SuggesterTest, TestLRUCache, DocValuesTest, TestLuceneMatchVersion, UpdateRequestProcessorFactoryTest, TestCollationFieldDocValues, CSVRequestHandlerTest, BinaryUpdateRequestHandlerTest, JsonLoaderTest, MoreLikeThisHandlerTest, TestGroupingSearch, RangeFacetTest, CoreMergeIndexesAdminHandlerTest, ShardRoutingTest, DistributedQueryComponentCustomSortTest, TestSearcherReuse, TestSurroundQueryParser, IndexSchemaTest, FunctionTest, LukeRequestHandlerTest, ExpressionTest, LoggingHandlerTest, TestFieldSortValues, DocumentBuilderTest, DOMUtilTest, UpdateParamsTest, MBeansHandlerTest, EchoParamsTest, DistributedTermsComponentTest, ResourceLoaderTest, SyncSliceTest, TestLMDirichletSimilarityFactory, TestReplicationHandler, TestManagedResourceStorage, FieldMutatingUpdateProcessorTest, TestRemoteStreaming, TriLevelCompositeIdRoutingTest, TestPostingsSolrHighlighter, TestFunctionQuery, TestSuggestSpellingConverter, ReturnFieldsTest, SpellPossibilityIteratorTest, DistributedQueryElevationComponentTest, StressHdfsTest, TestDocumentBuilder, SampleTest, SliceStateUpdateTest, SliceStateTest, HdfsBasicDistributedZkTest, TestJmxMonitoredMap, PeerSyncTest, LeaderElectionTest, TestFaceting, ClusterStateTest, SynonymTokenizerTest, SpatialFilterTest, ClusterStateUpdateTest, SOLR749Test, TestAddFieldRealTimeGet, TestPerFieldSimilarity, TestDocSet, SchemaVersionSpecificBehaviorTest, BadComponentTest, TestSolrDeletionPolicy1, CacheHeaderTest, SearchHandlerTest, OverseerTest, PolyFieldTest, CircularListTest, TestRealTimeGet, OverseerRolesTest, SolrTestCaseJ4Test, TestCharFilters, ZkCLITest, DeleteInactiveReplicaTest, QueryParsingTest, AliasIntegrationTest, TestSolrXMLSerializer, RecoveryZkTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.seed=CDB667E8A1E7CCF3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=Pacific/Port_Moresby -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | RecoveryZkTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([CDB667E8A1E7CCF3]:0)
[junit4] Completed on J0 in 7223.32s, 1 test, 2 errors <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 393 suites, 1651 tests, 1 suite-level error, 1 error, 49 ignored (28 assumptions)
Total time: 165 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build #
9910 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9910/
Java: 32bit/jdk1.7.0_51 -server -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([FA4A880D969CF294:7BAC0615E1C392A8]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 11621 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 42717 T147 oas.SolrTestCaseJ4.startTrackingSearchers WARN startTrackingSearchers: numOpens=6 numCloses=6
[junit4] 2> 42718 T147 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 42719 T147 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /asj/h
[junit4] 2> 42721 T147 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 42726 T147 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395820582019
[junit4] 2> 42730 T147 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 42733 T148 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 42832 T147 oasc.ZkTestServer.run start zk server on port:39266
[junit4] 2> 42867 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69316 T151 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 26426ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 69338 T149 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fd63d4b50000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 69341 T149 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 69344 T154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1114305 name:ZooKeeperConnection Watcher:127.0.0.1:39266 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69345 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69347 T147 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 69376 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69379 T156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ff834 name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69379 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69382 T147 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 69386 T147 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 69389 T147 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 69391 T147 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 69395 T147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 69395 T147 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 69437 T147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 69438 T147 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 69443 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 69443 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 69443 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 69444 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 69444 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 69444 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 69444 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 69445 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 69445 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 69449 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69451 T158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@242d75 name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69451 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69543 T147 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 69694 T147 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 69697 T147 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50566
[junit4] 2> 69699 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 69700 T147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 69700 T147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746
[junit4] 2> 69700 T147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746/'
[junit4] 2> 69712 T147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746/solr.xml
[junit4] 2> 69779 T147 oasc.CoreContainer.<init> New CoreContainer 17709234
[junit4] 2> 69780 T147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746/]
[junit4] 2> 69781 T147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 69782 T147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 69782 T147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 69782 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 69783 T147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 69783 T147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 69784 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 69784 T147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 69785 T147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 69789 T147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 69789 T147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 69789 T147 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 69790 T147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39266/solr
[junit4] 2> 69797 T147 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 69798 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69802 T169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1128e87 name:ZooKeeperConnection Watcher:127.0.0.1:39266 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69802 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69808 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69810 T171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@166de77 name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69810 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69817 T147 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 69823 T147 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 69834 T147 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 69839 T147 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 69846 T147 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 69858 T147 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 69861 T147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50566_asj%2Fh
[junit4] 2> 69864 T147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50566_asj%2Fh
[junit4] 2> 69929 T147 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 69932 T147 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 69945 T147 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50566_asj%2Fh
[junit4] 2> 69946 T147 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 69951 T147 oasc.Overseer.start Overseer (id=91476497666473990-127.0.0.1:50566_asj%2Fh-n_0000000000) starting
[junit4] 2> 69972 T147 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 70043 T173 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 70043 T147 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 70044 T173 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 70053 T147 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 70057 T147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 70068 T172 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 70077 T174 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 70077 T174 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 70081 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 70081 T174 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 70085 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 70086 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50566/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50566_asj%2Fh",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 70086 T172 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 70092 T172 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 70101 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 70111 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71082 T174 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 71082 T174 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 71083 T174 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 71083 T174 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 71085 T174 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 71085 T174 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746/collection1/'
[junit4] 2> 71132 T174 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 71181 T174 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 71184 T174 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 71195 T174 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 71524 T174 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 71527 T174 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 71528 T174 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 71528 T174 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 71529 T174 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 71529 T174 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 71529 T174 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395820608746/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/
[junit4] 2> 71529 T174 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 71530 T174 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data
[junit4] 2> 71530 T174 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/index/
[junit4] 2> 71530 T174 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 71530 T174 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/index
[junit4] 2> 71532 T174 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@a98388 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 71532 T174 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 71533 T174 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 71533 T174 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 71533 T174 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 71533 T174 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 71534 T174 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 71534 T174 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 71534 T174 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 71534 T174 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 71536 T174 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 71541 T174 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 71541 T174 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 71542 T174 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@a98388 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 71542 T174 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 71543 T174 oass.SolrIndexSearcher.<init> Opening Searcher@1c6bce9[collection1] main
[junit4] 2> 71543 T174 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 71544 T174 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 71544 T174 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 71547 T174 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 71547 T174 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 71547 T174 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 71548 T174 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 71548 T174 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 71548 T174 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 71548 T174 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 71549 T175 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c6bce9[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 71549 T174 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 71550 T178 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:50566/asj/h collection:control_collection shard:shard1
[junit4] 2> 71551 T147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 71551 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 71557 T178 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 71562 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71564 T180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d6ea name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71565 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71565 T178 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 71566 T147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 71568 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71569 T178 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 71569 T178 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 71570 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C43 name=collection1 org.apache.solr.core.SolrCore@52f59 url=https://127.0.0.1:50566/asj/h/collection1 node=127.0.0.1:50566_asj%2Fh C43_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:50566/asj/h, core=collection1, node_name=127.0.0.1:50566_asj%2Fh}
[junit4] 2> 71570 T178 C43 P50566 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:50566/asj/h/collection1/
[junit4] 2> 71571 T178 C43 P50566 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 71571 T178 C43 P50566 oasc.SyncStrategy.syncToMe https://127.0.0.1:50566/asj/h/collection1/ has no replicas
[junit4] 2> 71571 T178 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:50566/asj/h/collection1/ shard1
[junit4] 2> 71573 T178 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 71681 T147 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 71682 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71683 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71684 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71693 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 71697 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71736 T147 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 71737 T147 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 71743 T147 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 71746 T147 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37321
[junit4] 2> 71749 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 71750 T147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 71750 T147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974
[junit4] 2> 71751 T147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974/'
[junit4] 2> 71767 T147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974/solr.xml
[junit4] 2> 71800 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71800 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71820 T147 oasc.CoreContainer.<init> New CoreContainer 20483608
[junit4] 2> 71820 T147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974/]
[junit4] 2> 71821 T147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 71822 T147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 71822 T147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 71822 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 71823 T147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 71823 T147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 71823 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 71824 T147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 71824 T147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 71828 T147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 71828 T147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 71829 T147 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 71829 T147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39266/solr
[junit4] 2> 71830 T147 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 71830 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71833 T191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d40c2 name:ZooKeeperConnection Watcher:127.0.0.1:39266 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71833 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71836 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71837 T193 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13c354f name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71837 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71845 T178 oasc.ZkController.register We are https://127.0.0.1:50566/asj/h/collection1/ and leader is https://127.0.0.1:50566/asj/h/collection1/
[junit4] 2> 71845 T178 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50566/asj/h
[junit4] 2> 71845 T178 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 71846 T178 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 71846 T178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 71847 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71847 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71847 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71848 T178 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 71849 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 71851 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:50566/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50566_asj%2Fh",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 71854 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71856 T147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 71961 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71961 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71961 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72860 T147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37321_asj%2Fh
[junit4] 2> 72861 T147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37321_asj%2Fh
[junit4] 2> 72864 T171 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72864 T180 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72864 T193 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72873 T194 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 72873 T194 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 72874 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72874 T194 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 72874 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72874 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72875 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 72876 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37321/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37321_asj%2Fh",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 72877 T172 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 72877 T172 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 72880 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72984 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 72984 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 72984 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 73874 T194 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 73875 T194 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 73876 T194 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 73876 T194 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 73878 T194 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 73878 T194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974/collection1/'
[junit4] 2> 73942 T194 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 73983 T194 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 73986 T194 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 73999 T194 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 74325 T194 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 74327 T194 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 74329 T194 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 74329 T194 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 74329 T194 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 74330 T194 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 74330 T194 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395820610974/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/
[junit4] 2> 74330 T194 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 74330 T194 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1
[junit4] 2> 74331 T194 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/index/
[junit4] 2> 74331 T194 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 74331 T194 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/index
[junit4] 2> 74332 T194 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d27722 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 74332 T194 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74333 T194 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 74333 T194 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 74334 T194 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 74334 T194 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 74334 T194 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 74334 T194 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 74334 T194 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 74335 T194 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 74336 T194 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 74339 T194 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 74339 T194 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 74340 T194 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d27722 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 74340 T194 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74341 T194 oass.SolrIndexSearcher.<init> Opening Searcher@85be0[collection1] main
[junit4] 2> 74341 T194 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 74342 T194 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 74342 T194 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 74343 T194 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 74343 T194 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 74343 T194 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 74344 T194 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 74344 T194 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 74345 T194 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 74345 T194 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 74346 T195 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@85be0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 74346 T194 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 74347 T198 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:37321/asj/h collection:collection1 shard:shard2
[junit4] 2> 74347 T147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 74348 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 74348 T198 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 74358 T198 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 74360 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74360 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74360 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74362 T198 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 74362 T198 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C44 name=collection1 org.apache.solr.core.SolrCore@bcc3e9 url=https://127.0.0.1:37321/asj/h/collection1 node=127.0.0.1:37321_asj%2Fh C44_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:37321/asj/h, core=collection1, node_name=127.0.0.1:37321_asj%2Fh}
[junit4] 2> 74362 T198 C44 P37321 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:37321/asj/h/collection1/
[junit4] 2> 74362 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74362 T198 C44 P37321 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 74363 T198 C44 P37321 oasc.SyncStrategy.syncToMe https://127.0.0.1:37321/asj/h/collection1/ has no replicas
[junit4] 2> 74363 T198 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:37321/asj/h/collection1/ shard2
[junit4] 2> 74363 T198 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 74367 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74391 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74398 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74418 T147 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 74418 T147 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 74423 T147 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 74427 T147 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55446
[junit4] 2> 74430 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 74430 T147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 74431 T147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642
[junit4] 2> 74431 T147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642/'
[junit4] 2> 74451 T147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642/solr.xml
[junit4] 2> 74493 T147 oasc.CoreContainer.<init> New CoreContainer 4821315
[junit4] 2> 74494 T147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642/]
[junit4] 2> 74494 T147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 74495 T147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 74495 T147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 74495 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 74495 T147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 74496 T147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 74496 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 74496 T147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 74497 T147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 74500 T147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 74500 T147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 74500 T147 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 74500 T147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39266/solr
[junit4] 2> 74501 T147 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 74501 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74501 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74501 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74502 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74504 T209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18778c6 name:ZooKeeperConnection Watcher:127.0.0.1:39266 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74504 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74507 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74509 T211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1387da1 name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74509 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74519 T147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 74542 T198 oasc.ZkController.register We are https://127.0.0.1:37321/asj/h/collection1/ and leader is https://127.0.0.1:37321/asj/h/collection1/
[junit4] 2> 74543 T198 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37321/asj/h
[junit4] 2> 74543 T198 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 74543 T198 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 74544 T198 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 74547 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74548 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74548 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74549 T198 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74552 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74554 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:37321/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37321_asj%2Fh",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 74559 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74665 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74665 T211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74665 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74665 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 75523 T147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55446_asj%2Fh
[junit4] 2> 75525 T147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55446_asj%2Fh
[junit4] 2> 75528 T211 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75528 T193 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75528 T171 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75528 T180 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75538 T212 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 75538 T212 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 75540 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75540 T212 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 75540 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75540 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75542 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75543 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55446/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55446_asj%2Fh",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 75543 T172 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 75543 T172 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 75549 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75653 T211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75653 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75653 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75653 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 76540 T212 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 76540 T212 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 76541 T212 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 76541 T212 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 76543 T212 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 76543 T212 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642/collection1/'
[junit4] 2> 76576 T212 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 76611 T212 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 76612 T212 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 76621 T212 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 76940 T212 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 76942 T212 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 76944 T212 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 76944 T212 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 76944 T212 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 76945 T212 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 76945 T212 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395820613642/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/
[junit4] 2> 76945 T212 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 76945 T212 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2
[junit4] 2> 76946 T212 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/index/
[junit4] 2> 76946 T212 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 76946 T212 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/index
[junit4] 2> 76947 T212 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@da6066 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 76947 T212 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76948 T212 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 76949 T212 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 76949 T212 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 76949 T212 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 76949 T212 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 76949 T212 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 76950 T212 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 76950 T212 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 76952 T212 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 76954 T212 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 76954 T212 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 76955 T212 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@da6066 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 76955 T212 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76955 T212 oass.SolrIndexSearcher.<init> Opening Searcher@1e98963[collection1] main
[junit4] 2> 76956 T212 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 76957 T212 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 76957 T212 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 76958 T212 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 76958 T212 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 76959 T212 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 76959 T212 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 76960 T212 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 76960 T212 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 76960 T212 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 76961 T213 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e98963[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 76961 T212 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 76962 T216 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55446/asj/h collection:collection1 shard:shard1
[junit4] 2> 76962 T147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 76963 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 76963 T216 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 76970 T216 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 76972 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76972 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76972 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76972 T216 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 76973 T216 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C45 name=collection1 org.apache.solr.core.SolrCore@19752e1 url=https://127.0.0.1:55446/asj/h/collection1 node=127.0.0.1:55446_asj%2Fh C45_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55446/asj/h, core=collection1, node_name=127.0.0.1:55446_asj%2Fh}
[junit4] 2> 76973 T216 C45 P55446 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55446/asj/h/collection1/
[junit4] 2> 76973 T216 C45 P55446 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 76973 T216 C45 P55446 oasc.SyncStrategy.syncToMe https://127.0.0.1:55446/asj/h/collection1/ has no replicas
[junit4] 2> 76973 T216 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55446/asj/h/collection1/ shard1
[junit4] 2> 76974 T216 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 76976 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 76982 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76985 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76989 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77036 T147 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 77037 T147 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 77041 T147 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 77044 T147 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38364
[junit4] 2> 77046 T147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 77047 T147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 77047 T147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257
[junit4] 2> 77047 T147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257/'
[junit4] 2> 77058 T147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257/solr.xml
[junit4] 2> 77093 T211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77093 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77093 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77094 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77108 T147 oasc.CoreContainer.<init> New CoreContainer 6721172
[junit4] 2> 77109 T147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257/]
[junit4] 2> 77110 T147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 77110 T147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 77110 T147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 77110 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77111 T147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77111 T147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77111 T147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77111 T147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77112 T147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77120 T147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77120 T147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77120 T147 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77121 T147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39266/solr
[junit4] 2> 77121 T147 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77122 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77124 T227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16d56a5 name:ZooKeeperConnection Watcher:127.0.0.1:39266 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77124 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77127 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77128 T229 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f2fae4 name:ZooKeeperConnection Watcher:127.0.0.1:39266/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77129 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77136 T216 oasc.ZkController.register We are https://127.0.0.1:55446/asj/h/collection1/ and leader is https://127.0.0.1:55446/asj/h/collection1/
[junit4] 2> 77137 T216 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55446/asj/h
[junit4] 2> 77137 T216 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77137 T216 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 77137 T216 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77139 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77139 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77140 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77140 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77141 T147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 77141 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77142 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:55446/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55446_asj%2Fh",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 77149 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77255 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77255 T211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77255 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77255 T229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77255 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78151 T147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38364_asj%2Fh
[junit4] 2> 78152 T147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38364_asj%2Fh
[junit4] 2> 78155 T229 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78155 T171 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78155 T193 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78155 T211 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78155 T180 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78163 T230 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 78163 T230 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78164 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78164 T230 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78164 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78165 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78166 T172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78167 T172 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38364/asj/h",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38364_asj%2Fh",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78167 T172 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 78167 T172 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 78170 T171 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78272 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78273 T211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78273 T171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78273 T229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78273 T193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 79165 T230 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79165 T230 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79166 T230 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79166 T230 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79167 T230 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79167 T230 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257/collection1/'
[junit4] 2> 79200 T230 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 79236 T230 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79237 T230 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79245 T230 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79544 T230 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79547 T230 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79550 T230 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 79550 T230 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 79551 T230 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 79551 T230 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79551 T230 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395820616257/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/
[junit4] 2> 79552 T230 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 79552 T230 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3
[junit4] 2> 79552 T230 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index/
[junit4] 2> 79553 T230 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 79553 T230 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index
[junit4] 2> 79554 T230 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@15798c9 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 79555 T230 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79556 T230 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 79556 T230 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 79556 T230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 79556 T230 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 79556 T230 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 79557 T230 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 79557 T230 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 79557 T230 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 79559 T230 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 79561 T230 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 79561 T230 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 79562 T230 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@15798c9 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 79562 T230 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79562 T230 oass.SolrIndexSearcher.<init> Opening Searcher@1b31a82[collection1] main
[junit4] 2> 79563 T230 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79565 T230 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79565 T230 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 79566 T230 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 79566 T230 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 79566 T230 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 79567 T230 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 79567 T230 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 79568 T230 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 79568 T230 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 79569 T231 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b31a82[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 79569 T230 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 79570 T234 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38364/asj/h collection:collection1 shard:shard2
[junit4] 2> 79570 T147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-so
[...truncated too long message...]
87445 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 87445 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 87446 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 87446 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 87446 T147 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 87448 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 87449 T260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a20db3 name:ZooKeeperConnection Watcher:127.0.0.1:41238/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 87449 T147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 87453 T147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 87454 T261 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 87454 T261 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:465)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:457)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 88039 T171 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 88129 T193 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 88223 T211 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 88556 T261 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 88658 T147 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 88660 T147 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:41238 41238
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 88769 T147 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> ASYNC NEW_CORE C58 name=collection1 org.apache.solr.core.SolrCore@3c934b url=https://127.0.0.1:38364/asj/h/collection1 node=127.0.0.1:38364_asj%2Fh C58_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:38364/asj/h, core=collection1, node_name=127.0.0.1:38364_asj%2Fh}
[junit4] 2> 89137 T229 C58 P38364 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 89141 T229 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 89141 T229 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3c934b
[junit4] 2> 89142 T229 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=4}
[junit4] 2> 89142 T229 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 89142 T229 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 89142 T229 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 89143 T229 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 89144 T229 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 89144 T229 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3;done=false>>]
[junit4] 2> 89144 T229 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3
[junit4] 2> 89144 T229 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index;done=false>>]
[junit4] 2> 89144 T229 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395820582019/jetty3/index
[junit4] 2> 89144 T229 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 89145 T229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 89145 T229 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 89145 T229 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=15), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=15)), sim=DefaultSimilarity, locale=en_IN, timezone=Etc/GMT-14
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_51 (32-bit)/cpus=8,threads=1,free=25868576,total=65142784
[junit4] 2> NOTE: All tests run in this JVM: [SolrPingTest, LargeVolumeEmbeddedTest, DocumentAnalysisResponseTest, TestDocumentObjectBinder, SolrExampleEmbeddedTest, ContentStreamTest, LargeVolumeJettyTest, TestCoreAdmin, ModifiableSolrParamsTest, SolrExampleStreamingBinaryTest, ClientUtilsTest, TestEmbeddedSolrServer, CloudSolrServerTest]
[junit4] Completed on J0 in 47.10s, 2 tests, 1 failure <<< FAILURES!
[...truncated 76 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 failure
Total time: 59 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_51 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build #
9909 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9909/
Java: 32bit/jdk1.7.0_51 -server -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59183 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59183 within 45000 ms
at __randomizedtesting.SeedInfo.seed([36C4E8B30829AAE1:B72266AB7F76CADD]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59183 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11648 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 7312 T30 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 7313 T30 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ehw/
[junit4] 2> 7315 T30 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 7329 T30 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395812776743
[junit4] 2> 7333 T30 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 7336 T31 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 7435 T30 oasc.ZkTestServer.run start zk server on port:59183
[junit4] 2> 7485 T30 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96462 T34 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 88957ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 96463 T30 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 96481 T32 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fcecbb690000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96484 T32 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96488 T32 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96491 T30 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59183 59183
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=36C4E8B30829AAE1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl -Dtests.timezone=Etc/GMT-14 -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 94.4s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59183 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([36C4E8B30829AAE1:B72266AB7F76CADD]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59183 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 101761 T30 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395812871175
[junit4] 2> 101763 T30 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 101763 T38 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 101863 T30 oasc.ZkTestServer.run start zk server on port:37438
[junit4] 2> 101865 T30 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 101932 T44 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@131ef93 name:ZooKeeperConnection Watcher:127.0.0.1:37438 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 101932 T30 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 101934 T30 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 101959 T30 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 101961 T46 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70f34a name:ZooKeeperConnection Watcher:127.0.0.1:37438/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 101962 T30 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 101965 T30 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 101970 T30 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 101974 T30 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 101977 T30 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 101981 T30 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 101989 T30 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 101995 T30 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 101996 T30 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 102000 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 102000 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 102001 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 102001 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 102002 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 102002 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 102003 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 102003 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 102004 T30 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 102013 T30 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 102015 T47 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 102015 T47 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:465)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:457)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 103118 T47 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 103221 T30 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 103224 T30 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37438 37438
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 103282 T30 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 95977 T29 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=nl, timezone=Etc/GMT-14
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_51 (32-bit)/cpus=8,threads=1,free=32860616,total=64880640
[junit4] 2> NOTE: All tests run in this JVM: [TestDocumentObjectBinder, TermsResponseTest, AnlysisResponseBaseTest, SolrParamTest, TestUpdateRequestCodec, TestRetryUtil, ClientUtilsTest, ModifiableSolrParamsTest, CloudSolrServerTest]
[junit4] Completed on J0 in 96.54s, 2 tests, 1 error <<< FAILURES!
[...truncated 106 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 57 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_51 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b10) -
Build # 9908 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9908/
Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:56982 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:56982 within 45000 ms
at __randomizedtesting.SeedInfo.seed([F5D5A1676D0BDB97:74332F7F1A54BBAB]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:56982 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11602 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 21317 T95 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 21317 T95 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 21319 T95 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 21322 T95 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395801260326
[junit4] 2> 21324 T95 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 21326 T96 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 21426 T95 oasc.ZkTestServer.run start zk server on port:56982
[junit4] 2> 21461 T95 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 114513 T99 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 93035ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 114514 T95 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 114522 T97 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fc3d01580000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 114536 T97 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 114536 T97 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 114537 T97 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 114538 T95 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56982 56982
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=F5D5A1676D0BDB97 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_MT -Dtests.timezone=Asia/Kamchatka -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 93.5s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:56982 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([F5D5A1676D0BDB97:74332F7F1A54BBAB]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:56982 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 114782 T95 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395801353787
[junit4] 2> 114783 T95 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 114783 T103 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 114883 T95 oasc.ZkTestServer.run start zk server on port:40054
[junit4] 2> 114884 T95 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 114964 T109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dcdbf7 name:ZooKeeperConnection Watcher:127.0.0.1:40054 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 114964 T95 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 114966 T95 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 114988 T95 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 114991 T111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b27a08 name:ZooKeeperConnection Watcher:127.0.0.1:40054/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 114991 T95 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 114996 T95 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 115000 T95 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 115002 T95 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 115005 T95 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 115009 T95 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 115019 T95 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 115026 T95 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 115027 T95 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 115037 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 115037 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 115037 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 115038 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 115038 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 115038 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 115039 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 115039 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 115039 T95 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 115048 T95 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 115059 T112 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 115059 T112 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:465)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:457)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 116351 T112 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 116454 T95 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 116455 T95 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40054 40054
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 116514 T95 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 95204 T94 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=en_MT, timezone=Asia/Kamchatka
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=38861720,total=192413696
[junit4] 2> NOTE: All tests run in this JVM: [TestDocumentObjectBinder, AnlysisResponseBaseTest, SolrExampleXMLTest, SolrExampleEmbeddedTest, ContentStreamTest, ModifiableSolrParamsTest, TestUpdateRequestCodec, LargeVolumeJettyTest, LargeVolumeBinaryJettyTest, CloudSolrServerTest]
[junit4] Completed on J0 in 95.74s, 2 tests, 1 error <<< FAILURES!
[...truncated 97 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 57 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0) - Build # 9907 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9907/
Java: 32bit/jdk1.8.0 -server -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59004 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59004 within 45000 ms
at __randomizedtesting.SeedInfo.seed([10D89F0D312B7E32:913E111546741E0E]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59004 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11648 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 28231 T130 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 28231 T130 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /rea/
[junit4] 2> 28233 T130 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 28237 T130 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395793252856
[junit4] 2> 28239 T130 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 28241 T131 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 28341 T130 oasc.ZkTestServer.run start zk server on port:59004
[junit4] 2> 28377 T130 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 87896 T134 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 59499ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 87897 T130 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 87915 T132 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fbc2d23b0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 87926 T130 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59004 59004
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=10D89F0D312B7E32 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CR -Dtests.timezone=Asia/Dubai -Dtests.file.encoding=UTF-8
[junit4] ERROR 59.8s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59004 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([10D89F0D312B7E32:913E111546741E0E]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:59004 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 88062 T130 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395793312681
[junit4] 2> 88062 T130 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 88063 T138 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 88163 T130 oasc.ZkTestServer.run start zk server on port:54733
[junit4] 2> 88164 T130 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 88219 T144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bd603f name:ZooKeeperConnection Watcher:127.0.0.1:54733 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 88219 T130 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 88221 T130 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 88245 T130 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 88247 T146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e946f0 name:ZooKeeperConnection Watcher:127.0.0.1:54733/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 88247 T130 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 88251 T130 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 88255 T130 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 88258 T130 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 88263 T130 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 88269 T130 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 88270 T130 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 88277 T130 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 88278 T130 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 88281 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 88282 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 88282 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 88283 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 88283 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 88284 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 88284 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 88284 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 88285 T130 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 88294 T130 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 88298 T147 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 88299 T147 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:435)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:427)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:643)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 89401 T147 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 89504 T130 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 89506 T130 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54733 54733
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 89592 T130 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 61367 T129 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=es_CR, timezone=Asia/Dubai
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=1,free=166895408,total=214958080
[junit4] 2> NOTE: All tests run in this JVM: [DocumentAnalysisResponseTest, SolrExampleStreamingBinaryTest, TestXMLEscaping, TestCoreAdmin, TestDocumentObjectBinder, NoOpResponseParserTest, TestUpdateRequestCodec, SolrExceptionTest, TestJavaBinCodec, LargeVolumeEmbeddedTest, JettyWebappTest, AnlysisResponseBaseTest, CloudSolrServerTest]
[junit4] Completed on J0 in 62.40s, 2 tests, 1 error <<< FAILURES!
[...truncated 94 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 56 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b10) -
Build # 9906 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9906/
Java: 64bit/jdk1.7.0_60-ea-b10 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([2A4390E395B3481F:2E4B1F108716A73E]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:135)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:156)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:156)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:667)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:431)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:428)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:385)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:372)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:127)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:122)
... 45 more
Build Log:
[...truncated 10150 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 61870 T509 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 61870 T509 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1395785362263
[junit4] 2> 61871 T509 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 61875 T509 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 61876 T509 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 61876 T510 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 61976 T509 oasc.ZkTestServer.run start zk server on port:53640
[junit4] 2> 61978 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62343 T516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a3709d7 name:ZooKeeperConnection Watcher:127.0.0.1:53640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62344 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62345 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62351 T518 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30f9c5d5 name:ZooKeeperConnection Watcher:127.0.0.1:53640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62352 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62354 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62356 T520 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f72cca8 name:ZooKeeperConnection Watcher:127.0.0.1:53640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62356 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62357 T509 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 62360 T509 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 62364 T509 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 62367 T509 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 62369 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62373 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 62378 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62379 T522 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c71d0bd name:ZooKeeperConnection Watcher:127.0.0.1:53640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62379 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62380 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 62382 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 62388 T509 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:53640_solr
[junit4] 2> 62388 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 62390 T509 oasc.Overseer.start Overseer (id=91474189501792259-127.0.0.1:53640_solr-n_0000000000) starting
[junit4] 2> 62391 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 62395 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 62397 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 62399 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 62403 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 62409 T524 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 62410 T524 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 62411 T523 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 62412 T523 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 62412 T523 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 62413 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 62417 T523 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 62418 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 62419 T523 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 62421 T523 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 62422 T523 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 62424 T522 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 62424 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 62515 T509 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53640 53640
[junit4] 2> 62586 T509 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 62596 T509 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 62597 T509 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 62597 T525 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 62697 T509 oasc.ZkTestServer.run start zk server on port:41760
[junit4] 2> 62698 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62713 T531 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72b4b408 name:ZooKeeperConnection Watcher:127.0.0.1:41760 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62713 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62730 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62746 T533 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50ffdfc5 name:ZooKeeperConnection Watcher:127.0.0.1:41760 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62746 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62746 T509 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 62759 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62761 T535 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34e605d9 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62761 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62761 T509 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 62764 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62765 T537 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50bcc833 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62765 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62768 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 62770 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 62774 T509 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41760_solr
[junit4] 2> 62774 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 62776 T509 oasc.Overseer.start Overseer (id=91474189549305859-127.0.0.1:41760_solr-n_0000000000) starting
[junit4] 2> 62778 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 62785 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 62788 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 62791 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 62802 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 62806 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 62811 T539 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 62811 T509 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 62812 T538 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 62812 T539 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 62814 T509 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 62816 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62819 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62821 T541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47f9bd26 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62821 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62822 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62825 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 62827 T535 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 62827 T541 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 62828 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62830 T543 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a7d434d name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62830 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62832 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62834 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 62838 T535 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 62838 T543 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 62838 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62838 T541 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 62840 T545 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@248eba46 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62840 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62842 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62844 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 62848 T535 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 62848 T541 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 62848 T543 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 62848 T545 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 62849 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62850 T547 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e25e0d8 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62850 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62851 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62854 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 62857 T545 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 62857 T543 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 62857 T535 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 62857 T547 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 62857 T541 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 62858 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62859 T549 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c46465f name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62860 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62861 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62863 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 62866 T543 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62866 T541 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62867 T535 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62867 T545 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62868 T547 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62880 T549 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 62880 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62882 T551 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32ae8db4 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62882 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62884 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62886 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 62889 T543 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62889 T541 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62890 T547 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62890 T535 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62890 T545 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62893 T549 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62893 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62894 T551 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 62895 T553 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a47c8ea name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62895 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62897 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62899 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 62902 T545 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62902 T549 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62902 T543 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62903 T547 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62903 T535 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62903 T541 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62903 T553 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62903 T551 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 62904 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62905 T555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56540233 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62906 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62907 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62910 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 62914 T547 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62914 T545 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62916 T549 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62916 T551 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62915 T553 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62915 T543 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62917 T555 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62917 T541 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62917 T535 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 62918 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62919 T557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e024652 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62920 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62921 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62924 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 62927 T545 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62927 T551 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62927 T547 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62927 T543 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62928 T535 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62928 T541 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62928 T549 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62928 T555 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62928 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62929 T553 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62929 T557 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 62932 T559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54680b15 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62935 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62936 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62941 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 62944 T535 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62944 T543 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62946 T547 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62946 T541 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62947 T545 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62947 T553 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62947 T551 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62948 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62950 T561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e9de2af name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62950 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62951 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62953 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 62947 T557 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62950 T559 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62948 T555 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62947 T549 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 62957 T555 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T551 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T541 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T557 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T549 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T545 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62959 T561 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62959 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62959 T543 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T535 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T559 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62958 T553 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62961 T563 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19015800 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62958 T547 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 62961 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62962 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62964 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 62968 T545 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62968 T549 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62968 T557 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62969 T543 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62969 T559 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62969 T555 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62969 T553 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62970 T547 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62970 T551 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62969 T541 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62970 T561 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62970 T535 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62971 T563 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 62974 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62974 T565 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20a01f5b name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62975 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62976 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62979 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 62983 T551 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62983 T535 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62983 T555 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62983 T549 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T559 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62983 T547 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T553 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T565 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T545 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62983 T543 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T561 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T541 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62984 T557 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62985 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62985 T563 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 62987 T567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f770091 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 62988 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 62989 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 62991 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 62995 T547 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62995 T549 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62995 T541 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62995 T535 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62996 T555 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62996 T561 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62996 T551 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62996 T567 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62997 T553 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62997 T557 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62997 T559 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62998 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 62998 T543 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62998 T565 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62998 T563 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 62998 T545 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 63000 T569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@318f05ef name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63000 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63001 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63003 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 63008 T547 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63008 T549 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63009 T561 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63009 T543 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63009 T551 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63009 T535 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63009 T569 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63010 T567 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63010 T541 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63010 T555 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63010 T553 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63011 T559 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63011 T557 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63011 T563 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63011 T545 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63011 T565 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 63012 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63014 T571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58e14ffc name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63014 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63015 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63017 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 63021 T535 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T549 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T561 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T553 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T565 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T571 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T551 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T543 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T555 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T563 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T557 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T545 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T547 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63022 T559 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63025 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63024 T569 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63024 T567 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63023 T541 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 63026 T573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e8cc1fe name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63027 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63028 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63030 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 63034 T561 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63034 T547 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T535 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T541 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T551 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T549 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T573 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63035 T563 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63036 T571 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63036 T553 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63037 T555 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63037 T567 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63038 T559 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63038 T557 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63040 T545 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63040 T543 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63040 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63042 T565 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63042 T575 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30bc0e10 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63042 T569 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 63042 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63044 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63047 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 63052 T535 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63052 T555 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63053 T561 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63053 T567 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63053 T559 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63054 T547 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63054 T557 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63055 T545 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63055 T563 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63055 T553 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63056 T551 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63057 T549 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63057 T541 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63055 T565 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63057 T543 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63057 T573 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63058 T571 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63058 T575 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63060 T569 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 63071 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63071 T577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12e12b4 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63072 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63073 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63075 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 63079 T549 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63080 T561 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63081 T555 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T547 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T543 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T559 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63083 T573 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63080 T545 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63080 T535 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63079 T567 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63079 T541 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63079 T565 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63084 T571 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63084 T575 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63084 T577 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63083 T557 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63083 T569 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T551 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T553 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63082 T563 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 63092 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63095 T579 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65fa805 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63095 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63101 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63104 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 63108 T535 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63108 T545 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63109 T557 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63109 T553 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63110 T569 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63108 T565 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63108 T577 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63110 T573 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T567 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T563 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T575 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T571 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63110 T559 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63110 T551 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63109 T543 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63109 T555 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T541 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T561 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T547 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63111 T549 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63117 T579 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 63125 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63127 T581 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50d04b3e name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63127 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63128 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63131 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 63143 T535 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T547 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T553 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63143 T555 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T545 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T577 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T579 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T563 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T561 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63146 T567 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T541 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T551 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T565 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63144 T569 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63146 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63146 T549 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63146 T581 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63146 T571 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63146 T575 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T573 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T559 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T543 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63145 T557 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 63149 T583 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fe4801d name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63149 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63151 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63153 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 63158 T563 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63158 T561 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63158 T559 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63158 T555 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T541 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63163 T545 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63163 T573 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T547 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T565 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T543 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T557 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T569 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63162 T553 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63161 T577 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63161 T579 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63161 T551 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63161 T571 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63159 T581 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63159 T567 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63159 T549 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63159 T535 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63165 T585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ebd9110 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63163 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63163 T575 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63163 T583 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 63167 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63169 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63172 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 63178 T553 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63178 T571 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63179 T575 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63179 T547 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63178 T545 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63178 T569 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T559 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T549 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T583 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T573 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63181 T557 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63181 T555 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63178 T581 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63181 T541 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63181 T567 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T535 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T565 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T579 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T563 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63180 T577 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63179 T561 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63179 T551 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63179 T543 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63184 T585 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 63187 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63187 T587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21b535ed name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63187 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63189 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63192 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 63197 T571 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63198 T535 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63198 T573 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63198 T547 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63198 T579 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T565 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T551 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63200 T545 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63200 T577 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63200 T563 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63201 T555 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63201 T553 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63202 T557 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63202 T583 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63203 T549 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T581 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T569 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63203 T561 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63203 T585 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63202 T559 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63202 T587 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63201 T543 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63200 T567 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T541 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63199 T575 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 63207 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63208 T589 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c354da1 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63208 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63209 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63212 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 63217 T567 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63218 T543 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63218 T555 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63218 T585 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63217 T569 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63218 T541 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63218 T583 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63219 T587 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63219 T557 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T545 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T565 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T553 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T547 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T559 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T573 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T563 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T577 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63222 T561 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63222 T535 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T589 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63220 T579 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63222 T551 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63221 T581 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63221 T575 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63223 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63221 T549 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63223 T571 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 63224 T591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33b0d0c5 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63224 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63225 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63227 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 63232 T587 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63232 T563 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63232 T577 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63232 T549 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T555 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T569 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T565 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T551 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T561 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T573 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T575 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T571 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T589 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T553 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T591 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T535 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T543 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T567 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63234 T545 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T583 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63233 T557 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63236 T585 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63236 T541 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63236 T547 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63235 T579 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63235 T581 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63235 T559 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 63238 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63241 T593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4197d0de name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63241 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63242 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63245 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 63250 T579 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63250 T545 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63251 T567 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63251 T569 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T535 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T581 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T565 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T573 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T541 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63252 T547 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63253 T563 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63253 T553 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63253 T589 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63254 T561 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63253 T559 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63254 T585 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63254 T583 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63253 T557 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63255 T591 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63255 T549 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63255 T551 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63256 T575 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63256 T577 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63257 T593 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63258 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63259 T595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fa0d548 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63259 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63259 T543 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63259 T555 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63259 T571 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63259 T587 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 63260 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63263 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 63268 T585 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T555 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T543 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T559 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T587 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T583 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T563 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T575 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T553 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T561 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63271 T569 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T589 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63269 T591 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63272 T567 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63272 T541 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63271 T593 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63271 T579 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T557 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T549 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63273 T565 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63273 T571 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T577 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63270 T551 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63273 T573 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63273 T535 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63273 T545 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63272 T595 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63272 T581 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63274 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63274 T547 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 63276 T597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35926cb6 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63276 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63277 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63279 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 63285 T559 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63285 T589 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T563 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T583 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T585 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T597 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T581 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T553 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T569 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T587 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T579 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T561 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T577 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T595 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T549 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T545 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T565 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T547 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T593 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T535 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T567 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T571 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T541 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63291 T573 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63287 T557 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63287 T555 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63287 T543 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63286 T591 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T551 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63292 T575 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 63307 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63309 T599 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e5a75b5 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63309 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63310 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63316 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 63406 T551 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63406 T577 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63406 T595 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63406 T553 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63408 T573 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63408 T589 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63408 T569 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63408 T591 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T547 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T597 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T543 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T563 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T585 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T587 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T571 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T575 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63411 T579 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63411 T593 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T557 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T559 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T565 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63409 T581 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63411 T545 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63411 T561 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63411 T535 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T549 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T555 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T541 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T583 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T599 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63410 T567 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 63427 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63427 T601 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a711356 name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63428 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63429 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63431 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 63436 T567 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63436 T569 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63436 T535 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63437 T593 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63437 T545 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63438 T561 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63438 T549 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63439 T591 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63439 T577 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63439 T547 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T595 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T551 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T579 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T573 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63441 T571 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63441 T541 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T555 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63441 T565 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63441 T581 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63440 T553 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63443 T557 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63444 T563 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63444 T589 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63443 T597 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63444 T575 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63444 T585 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63444 T583 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63443 T559 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63445 T599 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63445 T587 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63445 T543 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63445 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63453 T601 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 63458 T603 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16c6174c name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63459 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63460 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63463 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 63473 T545 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63473 T569 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63473 T567 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63473 T565 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63474 T593 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63474 T541 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63474 T573 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63475 T571 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63474 T547 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63474 T579 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63475 T587 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63475 T583 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63475 T581 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63476 T557 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63476 T535 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63476 T559 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63476 T553 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63476 T591 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63477 T585 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63477 T543 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63477 T555 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T589 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T599 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63477 T603 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63477 T597 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T575 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63479 T601 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T577 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T595 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T549 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63478 T563 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63479 T561 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63479 T551 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 63480 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 63482 T605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38b6cf3d name:ZooKeeperConnection Watcher:127.0.0.1:41760/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 63482 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 63483 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 63485 T509 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 63491 T547 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T541 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T535 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T579 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T569 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T573 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T605 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T577 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T571 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63491 T581 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T599 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T589 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T595 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T563 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T549 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63494 T587 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63493 T561 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T601 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T575 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T565 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T551 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T567 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 63492 T545 oascc.ZkStateReader$3.process Updating
[...truncated too long message...]
atcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112229 T1062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 112229 T1056 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 112276 T1062 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 112279 T509 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:42927 42927
[junit4] 2> 112530 T509 oas.SolrTestCaseJ4.tearDown ###Ending testBadQueueItem
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 112537 T509 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 112537 T509 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 112537 T1063 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 112637 T509 oasc.ZkTestServer.run start zk server on port:41204
[junit4] 2> 112638 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 112642 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43f88247 name:ZooKeeperConnection Watcher:127.0.0.1:41204/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 112643 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 112643 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 112645 T1071 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9cc6d29 name:ZooKeeperConnection Watcher:127.0.0.1:41204 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 112645 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 112669 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 112671 T1073 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@397d29ec name:ZooKeeperConnection Watcher:127.0.0.1:41204 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 112671 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 112671 T509 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 112673 T509 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 112674 T509 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 112675 T509 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 112677 T509 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 112679 T509 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 112681 T1075 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f891438 name:ZooKeeperConnection Watcher:127.0.0.1:41204/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 112681 T509 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 112682 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 112683 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 112686 T509 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41204_solr
[junit4] 2> 112686 T509 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 112687 T509 oasc.Overseer.start Overseer (id=91474192824729603-127.0.0.1:41204_solr-n_0000000000) starting
[junit4] 2> 112688 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 112690 T509 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 112693 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 112694 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 112697 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 112699 T509 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 112703 T1077 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 112704 T1077 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 112704 T1076 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 112704 T509 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112706 T1076 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112706 T1076 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 112706 T1076 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 112708 T1075 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112709 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 112755 T509 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112757 T1075 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112758 T1076 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112759 T1076 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 112759 T1076 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 112759 T1076 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 112761 T1075 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112864 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 112911 T509 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:41204 41204
[junit4] 2> 113079 T509 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 113081 T509 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 113081 T509 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 116081 T509 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=bg_BG, timezone=America/Argentina/Buenos_Aires
[junit4] 2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=115328232,total=284164096
[junit4] 2> NOTE: All tests run in this JVM: [TestManagedSchemaFieldResource, SearchHandlerTest, TestSolrXmlPersistor, TestManagedStopFilterFactory, SolrIndexSplitterTest, AliasIntegrationTest, ReturnFieldsTest, DistributedQueryElevationComponentTest, TestDefaultSearchFieldResource, DistributedTermsComponentTest, OverseerTest]
[junit4] Completed on J0 in 54.25s, 9 tests, 1 error <<< FAILURES!
[...truncated 1091 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 393 suites, 1651 tests, 1 error, 49 ignored (28 assumptions)
Total time: 49 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b10 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b10) -
Build # 9904 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9904/
Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:52972 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:52972 within 45000 ms
at __randomizedtesting.SeedInfo.seed([E6F891076BD51336:58E98928CAF8644]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:52972 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11628 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 30306 T143 oas.SolrTestCaseJ4.startTrackingSearchers WARN startTrackingSearchers: numOpens=17 numCloses=17
[junit4] 2> 30306 T143 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 30307 T143 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 30310 T143 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 30313 T143 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395772696955
[junit4] 2> 30316 T143 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 30318 T144 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 30418 T143 oasc.ZkTestServer.run start zk server on port:52972
[junit4] 2> 30457 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111109 T147 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 80633ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 111111 T143 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 111129 T145 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fa8929b00000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 111131 T145 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fa8929b00002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 111132 T145 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fa8929b00003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 111133 T145 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fa8929b00004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 111136 T143 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52972 52972
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=E6F891076BD51336 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hu_HU -Dtests.timezone=WET -Dtests.file.encoding=UTF-8
[junit4] ERROR 81.0s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:52972 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([E6F891076BD51336:58E98928CAF8644]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:52972 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 111334 T143 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395772777976
[junit4] 2> 111335 T143 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 111335 T151 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 111435 T143 oasc.ZkTestServer.run start zk server on port:40963
[junit4] 2> 111436 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111448 T157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f03c14 name:ZooKeeperConnection Watcher:127.0.0.1:40963 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111449 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111450 T143 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 111477 T152 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144fa8a65fa0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 111477 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111479 T159 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63f864 name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111479 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111482 T143 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 111487 T143 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 111490 T143 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 111492 T143 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 111495 T143 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 111495 T143 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 111501 T143 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 111501 T143 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 111507 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 111507 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 111508 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 111508 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 111513 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 111513 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 111514 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 111514 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 111515 T143 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 111523 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111524 T161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2d985 name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111525 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111594 T143 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 111598 T143 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 111600 T143 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:47295
[junit4] 2> 111602 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 111602 T143 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 111602 T143 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170
[junit4] 2> 111603 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170/'
[junit4] 2> 111618 T143 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170/solr.xml
[junit4] 2> 111669 T143 oasc.CoreContainer.<init> New CoreContainer 16423974
[junit4] 2> 111669 T143 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170/]
[junit4] 2> 111670 T143 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 111670 T143 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 111671 T143 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 111671 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 111671 T143 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 111671 T143 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 111672 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 111672 T143 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 111672 T143 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 111675 T143 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 111675 T143 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 111675 T143 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 111676 T143 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40963/solr
[junit4] 2> 111685 T143 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 111685 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111687 T172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1760a96 name:ZooKeeperConnection Watcher:127.0.0.1:40963 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111688 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111693 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111695 T174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9b695 name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111695 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111705 T143 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 111710 T143 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 111714 T143 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 111718 T143 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 111721 T143 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 111728 T143 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 111731 T143 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47295_
[junit4] 2> 111733 T143 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47295_
[junit4] 2> 111736 T143 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 111799 T143 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 111810 T143 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47295_
[junit4] 2> 111810 T143 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 111814 T143 oasc.Overseer.start Overseer (id=91473364777566212-127.0.0.1:47295_-n_0000000000) starting
[junit4] 2> 111825 T143 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 111857 T176 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 111858 T143 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 111859 T176 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 111865 T143 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 111867 T143 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 111873 T175 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 111879 T177 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 111880 T177 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 111883 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 111883 T177 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 111886 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 111887 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:47295",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47295_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 111887 T175 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 111893 T175 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 111897 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 111901 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 112884 T177 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 112884 T177 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 112885 T177 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 112885 T177 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 112887 T177 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 112887 T177 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170/collection1/'
[junit4] 2> 112926 T177 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 112951 T177 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 112952 T177 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 112960 T177 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 112989 T177 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 112995 T177 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 113000 T177 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 113005 T177 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 113296 T177 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 113298 T177 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 113300 T177 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 113300 T177 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 113303 T177 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 113303 T177 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 113303 T177 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395772778170/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/
[junit4] 2> 113304 T177 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 113304 T177 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data
[junit4] 2> 113304 T177 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/index/
[junit4] 2> 113304 T177 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 113305 T177 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/index
[junit4] 2> 113306 T177 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2be308 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 113306 T177 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 113307 T177 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 113307 T177 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 113307 T177 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 113308 T177 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 113308 T177 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 113308 T177 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 113308 T177 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 113309 T177 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 113311 T177 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 113313 T177 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 113313 T177 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 113314 T177 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2be308 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 113315 T177 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 113315 T177 oass.SolrIndexSearcher.<init> Opening Searcher@1640c79[collection1] main
[junit4] 2> 113315 T177 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 113317 T177 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 113317 T177 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 113320 T177 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 113320 T177 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 113321 T177 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 113322 T177 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 113322 T177 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 113322 T177 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 113323 T177 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 113323 T178 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1640c79[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 113323 T177 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 113324 T181 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:47295 collection:control_collection shard:shard1
[junit4] 2> 113325 T143 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 113325 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 113331 T181 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 113331 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 113333 T183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14747a3 name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 113333 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 113334 T143 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 113339 T143 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 113339 T181 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 113342 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113343 T181 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 113343 T181 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 113343 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@33e9df url=https://127.0.0.1:47295/collection1 node=127.0.0.1:47295_ C24_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:47295, core=collection1, node_name=127.0.0.1:47295_}
[junit4] 2> 113344 T181 C24 P47295 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:47295/collection1/
[junit4] 2> 113345 T181 C24 P47295 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 113346 T181 C24 P47295 oasc.SyncStrategy.syncToMe https://127.0.0.1:47295/collection1/ has no replicas
[junit4] 2> 113346 T181 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:47295/collection1/ shard1
[junit4] 2> 113349 T181 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 113355 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113362 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113366 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113398 T143 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 113399 T143 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 113402 T143 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 113403 T143 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35092
[junit4] 2> 113405 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 113405 T143 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 113406 T143 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980
[junit4] 2> 113406 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980/'
[junit4] 2> 113420 T143 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980/solr.xml
[junit4] 2> 113450 T143 oasc.CoreContainer.<init> New CoreContainer 6608415
[junit4] 2> 113450 T143 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980/]
[junit4] 2> 113451 T143 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 113452 T143 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 113452 T143 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 113452 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 113453 T143 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 113453 T143 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 113453 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 113453 T143 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 113454 T143 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 113456 T143 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 113457 T143 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 113457 T143 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 113457 T143 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40963/solr
[junit4] 2> 113458 T143 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 113458 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 113461 T194 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d31d61 name:ZooKeeperConnection Watcher:127.0.0.1:40963 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 113461 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 113463 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 113464 T196 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1353d85 name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 113464 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 113467 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 113467 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 113478 T143 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 113513 T181 oasc.ZkController.register We are https://127.0.0.1:47295/collection1/ and leader is https://127.0.0.1:47295/collection1/
[junit4] 2> 113514 T181 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:47295
[junit4] 2> 113514 T181 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 113514 T181 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 113514 T181 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 113515 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113515 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113516 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113516 T181 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 113517 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 113518 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:47295",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47295_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 113521 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 113623 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 113623 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 113624 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 114483 T143 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35092_
[junit4] 2> 114485 T143 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35092_
[junit4] 2> 114488 T174 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 114488 T196 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 114488 T183 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 114495 T197 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 114496 T197 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 114497 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 114497 T197 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 114497 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 114498 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 114500 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 114500 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35092",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35092_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 114501 T175 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 114501 T175 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 114503 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 114607 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 114607 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 114607 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 115498 T197 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 115498 T197 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 115499 T197 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 115499 T197 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 115500 T197 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 115501 T197 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980/collection1/'
[junit4] 2> 115539 T197 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 115573 T197 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 115574 T197 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 115586 T197 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 115844 T197 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 115846 T197 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 115848 T197 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 115848 T197 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 115849 T197 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 115849 T197 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 115849 T197 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395772779980/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/
[junit4] 2> 115849 T197 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 115850 T197 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1
[junit4] 2> 115850 T197 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/index/
[junit4] 2> 115850 T197 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 115850 T197 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/index
[junit4] 2> 115852 T197 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7c7f14 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 115852 T197 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 115853 T197 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 115853 T197 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 115853 T197 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 115854 T197 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 115854 T197 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 115854 T197 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 115854 T197 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 115855 T197 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 115857 T197 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 115860 T197 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 115860 T197 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 115861 T197 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7c7f14 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 115861 T197 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 115861 T197 oass.SolrIndexSearcher.<init> Opening Searcher@127a21a[collection1] main
[junit4] 2> 115861 T197 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 115862 T197 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 115863 T197 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 115863 T197 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 115863 T197 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 115864 T197 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 115867 T197 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 115868 T197 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 115868 T197 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 115868 T197 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 115870 T198 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@127a21a[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 115870 T197 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 115871 T201 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35092 collection:collection1 shard:shard1
[junit4] 2> 115872 T143 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 115872 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 115875 T201 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 115883 T201 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 115886 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 115886 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 115886 T201 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 115886 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 115886 T201 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@1883f39 url=https://127.0.0.1:35092/collection1 node=127.0.0.1:35092_ C25_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:35092, core=collection1, node_name=127.0.0.1:35092_}
[junit4] 2> 115887 T201 C25 P35092 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35092/collection1/
[junit4] 2> 115887 T201 C25 P35092 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 115888 T201 C25 P35092 oasc.SyncStrategy.syncToMe https://127.0.0.1:35092/collection1/ has no replicas
[junit4] 2> 115888 T201 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35092/collection1/ shard1
[junit4] 2> 115888 T201 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 115889 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 115895 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 115901 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 115937 T143 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 115938 T143 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 115942 T143 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 115943 T143 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50957
[junit4] 2> 115945 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 115946 T143 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 115946 T143 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515
[junit4] 2> 115946 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515/'
[junit4] 2> 115958 T143 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515/solr.xml
[junit4] 2> 115988 T143 oasc.CoreContainer.<init> New CoreContainer 8610917
[junit4] 2> 115988 T143 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515/]
[junit4] 2> 115989 T143 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 115990 T143 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 115990 T143 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 115990 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 115991 T143 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 115991 T143 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 115991 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 115991 T143 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 115992 T143 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 115994 T143 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 115994 T143 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 115995 T143 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 115995 T143 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40963/solr
[junit4] 2> 115995 T143 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 115996 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 115998 T212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1182ea8 name:ZooKeeperConnection Watcher:127.0.0.1:40963 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 115998 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 116000 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 116107 T214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8b11e name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 116107 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 116109 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116109 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116109 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116115 T143 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 116147 T201 oasc.ZkController.register We are https://127.0.0.1:35092/collection1/ and leader is https://127.0.0.1:35092/collection1/
[junit4] 2> 116147 T201 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35092
[junit4] 2> 116147 T201 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 116147 T201 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 116147 T201 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 116149 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 116149 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 116149 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 116149 T201 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 116150 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 116151 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35092",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35092_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 116154 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 116256 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116256 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116256 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 116256 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 117119 T143 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50957_
[junit4] 2> 117121 T143 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50957_
[junit4] 2> 117126 T196 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 117126 T214 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 117126 T174 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 117127 T183 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 117141 T215 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 117141 T215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 117144 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 117144 T215 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 117145 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 117146 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 117148 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 117149 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50957",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50957_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 117150 T175 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 117151 T175 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 117155 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 117259 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 117260 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 117260 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 117259 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118145 T215 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 118146 T215 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 118147 T215 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 118148 T215 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 118150 T215 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 118150 T215 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515/collection1/'
[junit4] 2> 118182 T215 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 118221 T215 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 118223 T215 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 118230 T215 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 118474 T215 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 118477 T215 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 118478 T215 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 118479 T215 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 118479 T215 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 118479 T215 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 118480 T215 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395772782515/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/
[junit4] 2> 118480 T215 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 118480 T215 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2
[junit4] 2> 118481 T215 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index/
[junit4] 2> 118481 T215 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 118481 T215 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index
[junit4] 2> 118482 T215 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@c78e3a lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 118482 T215 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 118483 T215 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 118484 T215 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 118484 T215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 118484 T215 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 118484 T215 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 118485 T215 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 118485 T215 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 118485 T215 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 118487 T215 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 118490 T215 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 118490 T215 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 118491 T215 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@c78e3a lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 118491 T215 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 118491 T215 oass.SolrIndexSearcher.<init> Opening Searcher@1cc3776[collection1] main
[junit4] 2> 118492 T215 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 118493 T215 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 118494 T215 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 118494 T215 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 118495 T215 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 118495 T215 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 118496 T215 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 118496 T215 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 118496 T215 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 118496 T215 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 118497 T216 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cc3776[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 118497 T215 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 118498 T219 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:50957 collection:collection1 shard:shard2
[junit4] 2> 118498 T143 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 118498 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 118498 T219 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 118503 T219 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 118505 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118505 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118505 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118505 T219 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 118506 T219 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@1f3f11c url=https://127.0.0.1:50957/collection1 node=127.0.0.1:50957_ C26_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:50957, core=collection1, node_name=127.0.0.1:50957_}
[junit4] 2> 118506 T219 C26 P50957 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:50957/collection1/
[junit4] 2> 118506 T219 C26 P50957 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 118506 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 118506 T219 C26 P50957 oasc.SyncStrategy.syncToMe https://127.0.0.1:50957/collection1/ has no replicas
[junit4] 2> 118507 T219 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:50957/collection1/ shard2
[junit4] 2> 118507 T219 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 118509 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118511 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118514 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118550 T143 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 118550 T143 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 118556 T143 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 118558 T143 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:49088
[junit4] 2> 118560 T143 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 118560 T143 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 118561 T143 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395772785141
[junit4] 2> 118561 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395772785141/'
[junit4] 2> 118583 T143 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395772785141/solr.xml
[junit4] 2> 118621 T143 oasc.CoreContainer.<init> New CoreContainer 3282466
[junit4] 2> 118621 T143 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395772785141/]
[junit4] 2> 118622 T143 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 118624 T143 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 118624 T143 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 118624 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 118624 T143 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 118625 T143 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 118625 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118625 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118625 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118625 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118625 T143 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 118627 T143 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 118627 T143 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 118630 T143 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 118631 T143 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 118631 T143 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 118631 T143 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40963/solr
[junit4] 2> 118632 T143 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 118632 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 118634 T230 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f7da08 name:ZooKeeperConnection Watcher:127.0.0.1:40963 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 118634 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 118638 T143 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 118639 T232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19823ed name:ZooKeeperConnection Watcher:127.0.0.1:40963/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 118639 T143 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 118649 T143 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 118670 T219 oasc.ZkController.register We are https://127.0.0.1:50957/collection1/ and leader is https://127.0.0.1:50957/collection1/
[junit4] 2> 118670 T219 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50957
[junit4] 2> 118670 T219 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 118670 T219 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 118671 T219 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 118672 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118673 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118673 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118673 T219 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 118676 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 118677 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:50957",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50957_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 118681 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 118784 T232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118785 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118784 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118784 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 118785 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 119654 T143 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49088_
[junit4] 2> 119657 T143 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49088_
[junit4] 2> 119661 T196 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 119661 T183 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 119661 T174 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 119662 T232 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 119662 T214 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 119675 T233 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 119675 T233 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 119678 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 119678 T233 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 119678 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 119679 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 119682 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 119683 T175 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49088",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49088_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 119684 T175 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 119684 T175 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 119688 T174 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 119792 T196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 119792 T174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 119792 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 119792 T232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 119792 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 120679 T233 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 120680 T233 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 120681 T233 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 120681 T233 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 120683 T233 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 120684 T233 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395772785141/collection1/'
[junit4] 2> 120720 T233 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 120746 T233 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 120747 T233 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 120755 T233 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 121005 T233 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 121007 T233 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 121009 T233 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 121009 T233 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 121010 T233 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collec
[...truncated too long message...]
faultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 129559 T253 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 129561 T143 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 129561 T253 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50957",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50957_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 129562 T143 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 129563 T143 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2;done=false>>]
[junit4] 2> 129563 T143 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2
[junit4] 2> 129564 T143 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index;done=false>>]
[junit4] 2> 129564 T143 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty2/index
[junit4] 2> 129566 T214 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 129568 T214 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 129570 T183 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 129592 T143 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 129643 T143 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49088
[junit4] 2> 129644 T143 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=3282466
[junit4] 2> 129644 T143 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 129645 T143 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 129650 T183 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 129672 T143 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 129727 T143 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 47295
[junit4] 2> 129736 T143 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 129738 T143 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40963 40963
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 129887 T143 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 130975 T196 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 131984 T232 C27 P49088 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 131991 T232 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 131992 T232 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1a7bed0
[junit4] 2> 131993 T232 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 131993 T232 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 131994 T232 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 131994 T232 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 131997 T232 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 131997 T232 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 131997 T232 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3;done=false>>]
[junit4] 2> 131998 T232 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3
[junit4] 2> 131998 T232 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3/index;done=false>>]
[junit4] 2> 131998 T232 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395772777975/jetty3/index
[junit4] 2> 131999 T232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 131999 T232 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 132000 T232 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 132001 T232 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Lucene46: {text=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=FSTPulsing41), range_facet_sl=PostingsFormat(name=MockSep), range_facet_l=PostingsFormat(name=FSTPulsing41), multiDefault=PostingsFormat(name=NestedPulsing), intDefault=PostingsFormat(name=FSTPulsing41), timestamp=PostingsFormat(name=FSTPulsing41), range_facet_si=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Lucene41VarGapFixedInterval), a_t=PostingsFormat(name=FSTPulsing41)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=hu_HU, timezone=WET
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=90386648,total=185597952
[junit4] 2> NOTE: All tests run in this JVM: [SolrExampleStreamingBinaryTest, ModifiableSolrParamsTest, TestDocumentObjectBinder, LargeVolumeEmbeddedTest, ContentStreamTest, TestUpdateRequestCodec, LargeVolumeJettyTest, SolrExceptionTest, TermsResponseTest, DocumentAnalysisResponseTest, TestEmbeddedSolrServer, TestCoreAdmin, CloudSolrServerTest]
[junit4] Completed on J0 in 102.62s, 2 tests, 1 error <<< FAILURES!
[...truncated 94 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 64 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b10) -
Build # 9903 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9903/
Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerRolesTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=2720, name=Overseer-91472811719852040-127.0.0.1:53598_-n_0000000009, state=RUNNABLE, group=Overseer collection creation process.]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2720, name=Overseer-91472811719852040-127.0.0.1:53598_-n_0000000009, state=RUNNABLE, group=Overseer collection creation process.]
at __randomizedtesting.SeedInfo.seed([259116CF7F23797E:A47798D7087C1942]:0)
Caused by: org.apache.solr.common.cloud.ZooKeeperException:
at __randomizedtesting.SeedInfo.seed([259116CF7F23797E]:0)
at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:274)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000006
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:206)
at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:257)
... 1 more
Build Log:
[...truncated 10440 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
[junit4] 2> 468729 T2541 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 468732 T2541 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1395764338985
[junit4] 2> 468732 T2541 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 468733 T2542 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 468833 T2541 oasc.ZkTestServer.run start zk server on port:33190
[junit4] 2> 468834 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 468836 T2548 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c99e5e name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 468837 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 468837 T2541 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 468843 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 468843 T2550 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b8c1b name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 468843 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 468844 T2541 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 468846 T2541 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 468848 T2541 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 468849 T2541 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 468851 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 468852 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 468856 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 468857 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 468870 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 468871 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 468873 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 468874 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 468876 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 468877 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 468880 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 468880 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 468882 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 468882 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 468885 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 468885 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 468887 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 468888 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 468889 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 468890 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 468892 T2541 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 468893 T2541 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 469225 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 469228 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52962
[junit4] 2> 469229 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 469229 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 469230 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148
[junit4] 2> 469230 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148/'
[junit4] 2> 469268 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148/solr.xml
[junit4] 2> 469352 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148
[junit4] 2> 469353 T2541 oasc.CoreContainer.<init> New CoreContainer 29549445
[junit4] 2> 469353 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148/]
[junit4] 2> 469354 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 469354 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 469355 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 469355 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 469355 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 469355 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 469356 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 469356 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 469358 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 469359 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 469359 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 469359 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 469360 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 469360 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 469362 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 469371 T2561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e763ee name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 469371 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 469373 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 469374 T2563 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17e467e name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 469375 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 469376 T2541 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 469379 T2541 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 469381 T2541 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 469383 T2541 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 469387 T2541 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 469390 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 469391 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52962_
[junit4] 2> 469392 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52962_
[junit4] 2> 469402 T2541 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 469407 T2541 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 469410 T2541 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52962_
[junit4] 2> 469410 T2541 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 469411 T2541 oasc.Overseer.start Overseer (id=91472811719852035-127.0.0.1:52962_-n_0000000000) starting
[junit4] 2> 469413 T2541 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 469425 T2565 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 469425 T2541 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 469425 T2565 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 469428 T2541 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 469429 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 469431 T2564 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 469433 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148
[junit4] 2> 469442 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148/conf/
[junit4] 2> 469571 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 469572 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 469573 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 469573 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 469575 T2568 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c079d8 name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 469575 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 469576 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 469578 T2541 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 469898 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 469899 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 469901 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49072
[junit4] 2> 469902 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 469902 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 469903 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831
[junit4] 2> 469903 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831/'
[junit4] 2> 469930 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831/solr.xml
[junit4] 2> 470025 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831
[junit4] 2> 470025 T2541 oasc.CoreContainer.<init> New CoreContainer 15111961
[junit4] 2> 470026 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831/]
[junit4] 2> 470027 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 470027 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 470028 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 470028 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 470029 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 470029 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 470030 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 470030 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 470030 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 470031 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 470032 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 470032 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 470033 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 470033 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 470034 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 470035 T2579 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ca1c02 name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 470036 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 470038 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 470038 T2581 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14e61b2 name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 470039 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 470043 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 471045 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49072_
[junit4] 2> 471047 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49072_
[junit4] 2> 471050 T2563 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 471050 T2568 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 471050 T2581 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 471073 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831
[junit4] 2> 471087 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831/conf/
[junit4] 2> 471237 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 471238 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 471239 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 471513 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 471514 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 471517 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53598
[junit4] 2> 471519 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 471532 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 471533 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492
[junit4] 2> 471533 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492/'
[junit4] 2> 471554 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492/solr.xml
[junit4] 2> 471621 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492
[junit4] 2> 471622 T2541 oasc.CoreContainer.<init> New CoreContainer 10546173
[junit4] 2> 471622 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492/]
[junit4] 2> 471623 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 471624 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 471624 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 471624 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 471625 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 471625 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 471625 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 471626 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 471626 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 471626 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 471627 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 471627 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 471628 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 471628 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 471629 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 471630 T2593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e55344 name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 471631 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 471633 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 471634 T2595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12f0b0c name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 471634 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 471641 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 472643 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53598_
[junit4] 2> 472644 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53598_
[junit4] 2> 472646 T2563 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 472646 T2595 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 472646 T2568 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 472646 T2581 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 472651 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492
[junit4] 2> 472662 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492/conf/
[junit4] 2> 472803 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 472805 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 472805 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 473138 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 473139 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 473141 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51204
[junit4] 2> 473142 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 473142 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 473143 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059
[junit4] 2> 473143 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059/'
[junit4] 2> 473177 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059/solr.xml
[junit4] 2> 473275 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059
[junit4] 2> 473275 T2541 oasc.CoreContainer.<init> New CoreContainer 889790
[junit4] 2> 473276 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059/]
[junit4] 2> 473283 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 473283 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 473283 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 473284 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 473284 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 473284 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 473285 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 473285 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 473285 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 473286 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 473286 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 473286 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 473287 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 473287 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 473288 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 473289 T2607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15ebdbf name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 473289 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 473291 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 473292 T2609 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dfaea name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 473292 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 473296 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 474298 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51204_
[junit4] 2> 474300 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51204_
[junit4] 2> 474302 T2568 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 474302 T2609 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 474302 T2595 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 474302 T2563 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 474302 T2581 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 474306 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059
[junit4] 2> 474312 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059/conf/
[junit4] 2> 474468 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 474469 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 474470 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 474757 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 474758 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 474759 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43832
[junit4] 2> 474760 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 474760 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 474761 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723
[junit4] 2> 474761 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723/'
[junit4] 2> 474784 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723/solr.xml
[junit4] 2> 474868 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723
[junit4] 2> 474868 T2541 oasc.CoreContainer.<init> New CoreContainer 21843936
[junit4] 2> 474869 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723/]
[junit4] 2> 474869 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 474870 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 474870 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 474870 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 474871 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 474871 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 474871 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 474871 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 474872 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 474872 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 474873 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 474873 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 474873 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 474874 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 474874 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 474876 T2621 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b73b64 name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 474876 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 474878 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 474879 T2623 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ef5e7 name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 474879 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 474884 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 475886 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43832_
[junit4] 2> 475887 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43832_
[junit4] 2> 475889 T2568 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475890 T2609 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475890 T2595 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475890 T2563 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475889 T2581 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475890 T2623 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 475894 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723
[junit4] 2> 475910 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723/conf/
[junit4] 2> 476105 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 476106 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 476106 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 476419 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 5
[junit4] 2> 476420 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 476422 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49898
[junit4] 2> 476423 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 476423 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 476424 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360
[junit4] 2> 476424 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/'
[junit4] 2> 476450 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/solr.xml
[junit4] 2> 476551 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360
[junit4] 2> 476551 T2541 oasc.CoreContainer.<init> New CoreContainer 19202688
[junit4] 2> 476552 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/]
[junit4] 2> 476553 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 476553 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 476553 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 476554 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 476554 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 476554 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 476554 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 476555 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 476555 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 476556 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 476556 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 476556 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 476557 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 476557 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 476558 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 476559 T2635 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c73ab4 name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 476559 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 476561 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 476562 T2637 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b21e5 name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 476562 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 476566 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 477574 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49898_
[junit4] 2> 477575 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49898_
[junit4] 2> 477577 T2609 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477577 T2563 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477578 T2595 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477578 T2568 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477577 T2581 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477577 T2623 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477577 T2637 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 477591 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360
[junit4] 2> 477598 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/conf/
[junit4] 2> 477775 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 477776 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 477777 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 478111 T2541 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 6
[junit4] 2> 478111 T2541 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 478119 T2541 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39454
[junit4] 2> 478120 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 478120 T2541 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 478120 T2541 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030
[junit4] 2> 478129 T2541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/'
[junit4] 2> 478158 T2541 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/solr.xml
[junit4] 2> 478250 T2541 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030
[junit4] 2> 478251 T2541 oasc.CoreContainer.<init> New CoreContainer 6572456
[junit4] 2> 478251 T2541 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/]
[junit4] 2> 478252 T2541 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 478252 T2541 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 478253 T2541 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 478253 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 478253 T2541 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 478253 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 478254 T2541 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 478254 T2541 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 478254 T2541 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 478255 T2541 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 478269 T2541 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 478269 T2541 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 478270 T2541 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33190/solr
[junit4] 2> 478270 T2541 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 478275 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 478276 T2649 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd184 name:ZooKeeperConnection Watcher:127.0.0.1:33190 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 478277 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 478279 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 478280 T2651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a64666 name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 478280 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 478292 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 479301 T2541 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39454_
[junit4] 2> 479315 T2541 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39454_
[junit4] 2> 479317 T2609 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2563 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2623 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2637 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2581 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2595 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2651 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479317 T2568 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 479321 T2541 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030
[junit4] 2> 479326 T2541 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/conf/
[junit4] 2> 479457 T2541 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 479458 T2541 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 479458 T2541 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 479459 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 479461 T2654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c1bd3a name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 479461 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 479462 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 479467 T2541 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 479467 T2656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1abae9b name:ZooKeeperConnection Watcher:127.0.0.1:33190/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 479467 T2541 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 479468 T2541 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 479472 T2647 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=4&version=2&maxShardsPerNode=2&name=testOverseerCol&replicationFactor=2&wt=javabin&action=CREATE
[junit4] 2> 479477 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479478 T2565 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerCol",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 479478 T2565 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerCol",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 479479 T2565 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
[junit4] 2> 479479 T2565 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol
[junit4] 2> 479480 T2565 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
[junit4] 2> 479482 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479483 T2564 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 479483 T2564 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
[junit4] 2> 479484 T2564 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
[junit4] 2> 479485 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479486 T2637 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2623 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2568 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479486 T2581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479487 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479489 T2595 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479582 T2565 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection, shardNames [shard1, shard2, shard3, shard4] , replicationFactor : 2
[junit4] 2> 479583 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:49898_
[junit4] 2> 479584 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:51204_
[junit4] 2> 479585 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:52962_
[junit4] 2> 479600 T2630 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard1&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard1_replica1&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479601 T2603 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard1&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard1_replica2&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479601 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:43832_
[junit4] 2> 479602 T2558 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard2&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard2_replica1&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479603 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:39454_
[junit4] 2> 479603 T2603 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down collection=testOverseerCol
[junit4] 2> 479603 T2630 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down collection=testOverseerCol
[junit4] 2> 479604 T2617 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard2&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard2_replica2&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479605 T2558 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down collection=testOverseerCol
[junit4] 2> 479606 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479606 T2558 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479607 T2617 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down collection=testOverseerCol
[junit4] 2> 479607 T2564 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 479605 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:53598_
[junit4] 2> 479608 T2617 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479606 T2603 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479608 T2644 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard3&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard3_replica1&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479606 T2630 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479608 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:49072_
[junit4] 2> 479609 T2588 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard3&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard3_replica2&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479608 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51204",
[junit4] 2> "core":"testOverseerCol_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51204_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479608 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479611 T2565 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:49898_
[junit4] 2> 479612 T2577 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard4&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard4_replica1&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479613 T2644 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down collection=testOverseerCol
[junit4] 2> 479613 T2588 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down collection=testOverseerCol
[junit4] 2> 479614 T2577 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down collection=testOverseerCol
[junit4] 2> 479614 T2588 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479614 T2632 oasha.CoreAdminHandler.handleCreateAction core create command shard=shard4&qt=/admin/cores&numShards=4&version=2&collection=testOverseerCol&name=testOverseerCol_shard4_replica2&wt=javabin&collection.configName=conf1&action=CREATE
[junit4] 2> 479614 T2644 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479615 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49898",
[junit4] 2> "core":"testOverseerCol_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49898_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479615 T2577 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479617 T2632 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down collection=testOverseerCol
[junit4] 2> 479618 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479618 T2632 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 479618 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52962",
[junit4] 2> "core":"testOverseerCol_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52962_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479623 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479624 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43832",
[junit4] 2> "core":"testOverseerCol_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43832_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479626 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479627 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39454",
[junit4] 2> "core":"testOverseerCol_shard3_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39454_",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479629 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479629 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53598",
[junit4] 2> "core":"testOverseerCol_shard3_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53598_",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479632 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479633 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49072",
[junit4] 2> "core":"testOverseerCol_shard4_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49072_",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479635 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479635 T2564 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49898",
[junit4] 2> "core":"testOverseerCol_shard4_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49898_",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerCol",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 479637 T2563 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 479746 T2568 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2623 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479747 T2637 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479746 T2563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479746 T2595 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 479746 T2609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 480607 T2558 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
[junit4] 2> 480608 T2558 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480609 T2603 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
[junit4] 2> 480609 T2558 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480609 T2617 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
[junit4] 2> 480609 T2630 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
[junit4] 2> 480622 T2588 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
[junit4] 2> 480622 T2644 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
[junit4] 2> 480622 T2577 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
[junit4] 2> 480624 T2644 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480622 T2632 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
[junit4] 2> 480610 T2603 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480625 T2632 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480626 T2644 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480627 T2644 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480610 T2558 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480627 T2603 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480628 T2603 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480629 T2644 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480626 T2632 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480624 T2577 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480630 T2577 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480630 T2577 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480623 T2588 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480623 T2630 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480631 T2588 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480632 T2588 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480623 T2617 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
[junit4] 2> 480632 T2630 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480633 T2630 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480631 T2577 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480630 T2558 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480629 T2632 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480629 T2644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/'
[junit4] 2> 480629 T2603 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480636 T2603 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1395764343059/testOverseerCol_shard1_replica2/'
[junit4] 2> 480637 T2632 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480638 T2632 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/testOverseerCol_shard4_replica2/'
[junit4] 2> 480635 T2558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1395764339148/testOverseerCol_shard2_replica1/'
[junit4] 2> 480635 T2630 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480640 T2630 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1395764346360/testOverseerCol_shard1_replica1/'
[junit4] 2> 480635 T2577 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1395764339831/testOverseerCol_shard4_replica1/'
[junit4] 2> 480633 T2617 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 480633 T2588 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480642 T2617 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
[junit4] 2> 480642 T2588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1395764341492/testOverseerCol_shard3_replica2/'
[junit4] 2> 480643 T2617 oascc.ZkStateReader.readConfigName path=/collections/testOverseerCol configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 480644 T2617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1395764344723/testOverseerCol_shard2_replica2/'
[junit4] 2> 480906 T2644 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 480906 T2632 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481041 T2588 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481246 T2617 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481335 T2558 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481400 T2603 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481522 T2630 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481574 T2577 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 481621 T2644 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 481632 T2644 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 481697 T2588 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 481698 T2588 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 481789 T2632 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 481857 T2644 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
[junit4] 2> 481872 T2632 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 481903 T2588 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
[junit4] 2> 481918 T2558 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 481972 T2558 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 482010 T2632 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
[junit4] 2> 482089 T2558 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
[junit4] 2> 482196 T2603 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 482198 T2603 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 482247 T2630 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 482260 T2630 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 482260 T2617 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 482271 T2617 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 482285 T2577 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 482286 T2577 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 482358 T2577 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
[junit4] 2> 482359 T2603 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
[junit4] 2> 482373 T2630 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
[junit4] 2> 482375 T2617 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
[junit4] 2> 486786 T2644 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486820 T2644 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486834 T2644 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 486834 T2558 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486849 T2588 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486886 T2588 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486896 T2577 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486897 T2603 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486909 T2558 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486921 T2588 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 486922 T2603 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486921 T2558 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 486931 T2577 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486931 T2644 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 486943 T2577 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 486953 T2603 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 486954 T2644 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 486957 T2632 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 486978 T2644 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 486981 T2644 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 486982 T2644 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 486983 T2644 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 486992 T2644 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 486993 T2644 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 486993 T2632 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 486993 T2644 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica1' using configuration from collection testOverseerCol
[junit4] 2> 487003 T2644 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 487003 T2644 oasc.SolrCore.<init> [testOverseerCol_shard3_replica1] Opening new SolrCore at ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/, dataDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/data/
[junit4] 2> 487004 T2617 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 487004 T2644 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e2dfad
[junit4] 2> 487005 T2632 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 487005 T2588 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487006 T2644 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/data
[junit4] 2> 487007 T2644 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/data/index/
[junit4] 2> 487016 T2644 oasc.SolrCore.initIndex WARN [testOverseerCol_shard3_replica1] Solr index directory './org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 487017 T2644 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1395764348030/testOverseerCol_shard3_replica1/data/index
[junit4] 2> 487017 T2644 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@663d70
[junit4] 2> 487020 T2617 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 487020 T2577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487031 T2588 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487032 T2603 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487033 T2617 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 487032 T2558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487031 T2644 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@a49a04 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4b9093),segFN=segments_1,generation=1}
[junit4] 2> 487044 T2644 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 487047 T2588 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487044 T2577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487047 T2603 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487047 T2558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487048 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 487060 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 487060 T2588 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487061 T2644 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 487061 T2644 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 487062 T2644 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 487062 T2644 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 487063 T2577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487064 T2558 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487064 T2632 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487062 T2630 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 487072 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 487073 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 487064 T2577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 487074 T2577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 487075 T2577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4]
[...truncated too long message...]
_elect/shard2/election/91472811719852044-core_node4-n_0000000001
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:176)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:173)
[junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:71)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:309)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 501188 T2623 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 501188 T2623 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 501189 T2623 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4] 2> 501189 T2623 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 501189 T2623 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 501559 T2718 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 32893 T2540 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 501698 T2651 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 501699 T2651 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=63), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=63)), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_US, timezone=America/Chihuahua
[junit4] 2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=223444688,total=302010368
[junit4] 2> NOTE: All tests run in this JVM: [TestCharFilters, SolrInfoMBeanTest, TestCollationFieldDocValues, CustomCollectionTest, TestBadConfig, TestElisionMultitermQuery, RecoveryZkTest, TestFreeTextSuggestions, PrimitiveFieldTypeTest, SoftAutoCommitTest, TestSearcherReuse, DistributedExpandComponentTest, TestMiniSolrCloudCluster, RemoteQueryErrorTest, CoreMergeIndexesAdminHandlerTest, SolrCoreTest, TestPHPSerializedResponseWriter, TestSolrXml, SuggesterTest, TestMergePolicyConfig, QueryElevationComponentTest, TestPostingsSolrHighlighter, TestSchemaVersionResource, TestLFUCache, TestJmxMonitoredMap, TestAddFieldRealTimeGet, TestDefaultSearchFieldResource, ParsingFieldUpdateProcessorsTest, BadComponentTest, TestBM25SimilarityFactory, UpdateRequestProcessorFactoryTest, ResourceLoaderTest, TestConfig, TestHighlightDedupGrouping, TestSystemIdResolver, PeerSyncTest, TestImplicitCoreProperties, LukeRequestHandlerTest, CachingDirectoryFactoryTest, FastVectorHighlighterTest, SimpleFacetsTest, URLClassifyProcessorTest, AliasIntegrationTest, TestManagedSchemaFieldResource, TestSolrXmlPersistor, TestDistribDocBasedVersion, AutoCommitTest, SolrIndexSplitterTest, SpellPossibilityIteratorTest, WordBreakSolrSpellCheckerTest, TestSolrIndexConfig, ZkControllerTest, CoreAdminHandlerTest, CursorMarkTest, TestNonNRTOpen, TestCoreContainer, AbstractAnalyticsStatsTest, LeaderElectionTest, TestDocSet, TestLMJelinekMercerSimilarityFactory, DOMUtilTest, AddSchemaFieldsUpdateProcessorFactoryTest, DocumentBuilderTest, DistributedTermsComponentTest, SynonymTokenizerTest, OverseerTest, TestTrie, TestBlendedInfixSuggestions, DirectUpdateHandlerOptimizeTest, UpdateParamsTest, OverseerRolesTest]
[junit4] Completed on J0 in 33.21s, 1 test, 1 error <<< FAILURES!
[...truncated 836 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 393 suites, 1651 tests, 1 error, 49 ignored (28 assumptions)
Total time: 60 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b10) -
Build # 9902 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9902/
Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:42140 within 45000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:42140 within 45000 ms
at __randomizedtesting.SeedInfo.seed([213FB6EBBF3C5294:A0D938F3C86332A8]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:42140 within 45000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
... 45 more
Build Log:
[...truncated 11630 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 26246 T94 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 26246 T94 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 26248 T94 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 26252 T94 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395758043727
[junit4] 2> 26258 T94 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 26261 T95 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 26361 T94 oasc.ZkTestServer.run start zk server on port:42140
[junit4] 2> 26394 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96559 T98 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 70145ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 96560 T94 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 96573 T96 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144f9a992920000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96575 T96 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96576 T96 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96577 T96 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144f9a992920003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 96579 T94 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:42140 42140
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=213FB6EBBF3C5294 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh -Dtests.timezone=Etc/GMT-12 -Dtests.file.encoding=UTF-8
[junit4] ERROR 70.5s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:42140 within 45000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([213FB6EBBF3C5294:A0D938F3C86332A8]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:150)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:101)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:91)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:85)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:77)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:95)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:42140 within 45000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:142)
[junit4] > ... 45 more
[junit4] 2> 96710 T94 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395758114185
[junit4] 2> 96711 T94 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 96712 T102 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 96812 T94 oasc.ZkTestServer.run start zk server on port:54395
[junit4] 2> 96813 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96861 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e0c45f6 name:ZooKeeperConnection Watcher:127.0.0.1:54395 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 96862 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 96864 T94 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 96886 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96889 T110 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f4396ce name:ZooKeeperConnection Watcher:127.0.0.1:54395/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 96889 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 96892 T94 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 96897 T94 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 96900 T94 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 96903 T94 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 96907 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 96908 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 96915 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 96915 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 96929 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 96929 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 96930 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 96930 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 96930 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 96931 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 96931 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 96932 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 96932 T94 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 96938 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96943 T112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6788e58c name:ZooKeeperConnection Watcher:127.0.0.1:54395/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 96943 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 96954 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 97052 T113 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 97052 T113 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:465)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:457)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 98166 T113 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 98268 T94 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 98270 T94 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54395 54395
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 103313 T94 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=zh, timezone=Etc/GMT-12
[junit4] 2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=63816872,total=126418944
[junit4] 2> NOTE: All tests run in this JVM: [LargeVolumeEmbeddedTest, TestXMLEscaping, TestRetryUtil, ClientUtilsTest, TestDocumentObjectBinder, TestUpdateRequestCodec, SolrExampleXMLTest, SolrParamTest, TestJavaBinCodec, TestCoreAdmin, SolrExampleEmbeddedTest, SolrExceptionTest, CloudSolrServerTest]
[junit4] Completed on J0 in 77.11s, 2 tests, 1 error <<< FAILURES!
[...truncated 97 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 error
Total time: 60 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure