You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/06/18 04:48:22 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build #
10587 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10587/
Java: 32bit/jdk1.8.0_05 -server -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerRolesTest.testDistribSearch
Error Message:
could not set the new overseer . expected 127.0.0.1:46515_sb current order : [127.0.0.1:46515_sb, 127.0.0.1:40347_sb, 127.0.0.1:60816_sb, 127.0.0.1:41246_sb, 127.0.0.1:58213_sb, 127.0.0.1:35314_sb, 127.0.0.1:52144_sb] ldr :null
Stack Trace:
java.lang.AssertionError: could not set the new overseer . expected 127.0.0.1:46515_sb current order : [127.0.0.1:46515_sb, 127.0.0.1:40347_sb, 127.0.0.1:60816_sb, 127.0.0.1:41246_sb, 127.0.0.1:58213_sb, 127.0.0.1:35314_sb, 127.0.0.1:52144_sb] ldr :null
at __randomizedtesting.SeedInfo.seed([69AA10045826F286:E84C9E1C2F7992BA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.OverseerRolesTest.testOverseerRole(OverseerRolesTest.java:158)
at org.apache.solr.cloud.OverseerRolesTest.doTest(OverseerRolesTest.java:94)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11967 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/init-core-data-001
[junit4] 2> 1369885 T5706 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /sb/
[junit4] 2> 1369888 T5706 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1369889 T5706 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1369890 T5707 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1369989 T5706 oasc.ZkTestServer.run start zk server on port:50897
[junit4] 2> 1369991 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1370324 T5713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d49ca name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1370324 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1370325 T5706 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1370328 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1370330 T5715 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@90c7ad name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1370330 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1370330 T5706 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1370332 T5706 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1370333 T5706 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1370335 T5706 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1370337 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1370337 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1370340 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1370340 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1370342 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1370343 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1370346 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1370347 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1370351 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1370352 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1370357 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1370357 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1370359 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1370360 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1370362 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1370363 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1370364 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1370365 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1370368 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1370368 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1370370 T5706 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1370370 T5706 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1370804 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1370814 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40347
[junit4] 2> 1370815 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1370816 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1370817 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002
[junit4] 2> 1370818 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002/'
[junit4] 2> 1370832 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002/solr.xml
[junit4] 2> 1370841 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002
[junit4] 2> 1370841 T5706 oasc.CoreContainer.<init> New CoreContainer 6720971
[junit4] 2> 1370842 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002/]
[junit4] 2> 1370842 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1370843 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1370843 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1370844 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1370844 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1370844 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1370844 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1370845 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1370845 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1370846 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1370846 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1370846 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1370847 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1370847 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1370848 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1370849 T5726 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13e96fe name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1370850 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1370851 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1370852 T5728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@893752 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1370852 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1370854 T5706 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1370858 T5706 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1370861 T5706 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1370863 T5706 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1370865 T5706 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1370866 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1370867 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40347_sb
[junit4] 2> 1370868 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40347_sb
[junit4] 2> 1370872 T5706 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1370873 T5706 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1370876 T5706 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40347_sb
[junit4] 2> 1370877 T5706 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1370878 T5706 oasc.Overseer.start Overseer (id=91950912719552515-127.0.0.1:40347_sb-n_0000000000) starting
[junit4] 2> 1370880 T5706 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1370889 T5730 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1370892 T5706 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1370895 T5706 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1370897 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1370899 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002
[junit4] 2> 1370903 T5729 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1370909 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002/conf/
[junit4] 2> 1371073 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1371075 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1371076 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1371077 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1371078 T5733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fb94e8 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1371078 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1371079 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1371081 T5706 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1371413 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1371414 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1371417 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35314
[junit4] 2> 1371418 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1371418 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1371419 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003
[junit4] 2> 1371419 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003/'
[junit4] 2> 1371435 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003/solr.xml
[junit4] 2> 1371461 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003
[junit4] 2> 1371461 T5706 oasc.CoreContainer.<init> New CoreContainer 29038745
[junit4] 2> 1371462 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003/]
[junit4] 2> 1371463 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1371463 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1371463 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1371464 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1371464 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1371465 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1371465 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1371466 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1371466 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1371467 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1371467 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1371468 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1371468 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1371469 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1371469 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1371471 T5744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5b0a2 name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1371472 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1371474 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1371475 T5746 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@208034 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1371475 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1371480 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1372482 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35314_sb
[junit4] 2> 1372484 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35314_sb
[junit4] 2> 1372490 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003
[junit4] 2> 1372501 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003/conf/
[junit4] 2> 1372731 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1372733 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1372734 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1373139 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1373141 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1373181 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60816
[junit4] 2> 1373182 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1373183 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1373183 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004
[junit4] 2> 1373183 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/'
[junit4] 2> 1373200 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/solr.xml
[junit4] 2> 1373213 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004
[junit4] 2> 1373214 T5706 oasc.CoreContainer.<init> New CoreContainer 7901547
[junit4] 2> 1373215 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/]
[junit4] 2> 1373215 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1373216 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1373216 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1373217 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1373217 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1373217 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1373218 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1373218 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1373219 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1373220 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1373220 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1373220 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1373221 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1373221 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1373222 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1373223 T5758 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19d2643 name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1373224 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1373226 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1373227 T5760 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10de003 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1373227 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1373231 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1374233 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60816_sb
[junit4] 2> 1374234 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60816_sb
[junit4] 2> 1374242 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004
[junit4] 2> 1374254 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/conf/
[junit4] 2> 1374512 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1374513 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1374513 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1374811 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 1374812 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1374814 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58213
[junit4] 2> 1374815 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1374815 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1374816 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005
[junit4] 2> 1374816 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/'
[junit4] 2> 1374827 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/solr.xml
[junit4] 2> 1374835 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005
[junit4] 2> 1374835 T5706 oasc.CoreContainer.<init> New CoreContainer 21885272
[junit4] 2> 1374835 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/]
[junit4] 2> 1374836 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1374836 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1374837 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1374837 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1374837 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1374838 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1374838 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1374838 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1374839 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1374839 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1374840 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1374840 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1374840 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1374841 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1374841 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1374842 T5772 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1eb028e name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1374843 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1374845 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1374846 T5774 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35cb17 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1374846 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1374852 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1375854 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58213_sb
[junit4] 2> 1375856 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58213_sb
[junit4] 2> 1375860 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005
[junit4] 2> 1375866 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/conf/
[junit4] 2> 1375997 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1375998 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1375998 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1376230 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 1376230 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1376232 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41246
[junit4] 2> 1376233 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1376234 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1376234 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006
[junit4] 2> 1376234 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006/'
[junit4] 2> 1376246 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006/solr.xml
[junit4] 2> 1376265 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006
[junit4] 2> 1376265 T5706 oasc.CoreContainer.<init> New CoreContainer 28703591
[junit4] 2> 1376266 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006/]
[junit4] 2> 1376266 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1376266 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1376267 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1376267 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1376267 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1376268 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1376268 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1376268 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1376269 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1376269 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1376269 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1376270 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1376270 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1376270 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1376271 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1376272 T5786 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9cc5c5 name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1376273 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1376275 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1376275 T5788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a02296 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1376275 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1376279 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1377281 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41246_sb
[junit4] 2> 1377286 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41246_sb
[junit4] 2> 1377290 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006
[junit4] 2> 1377295 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006/conf/
[junit4] 2> 1377388 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1377389 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1377390 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1377624 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 5
[junit4] 2> 1377625 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1377626 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52144
[junit4] 2> 1377628 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1377628 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1377628 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007
[junit4] 2> 1377629 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007/'
[junit4] 2> 1377639 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007/solr.xml
[junit4] 2> 1377645 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007
[junit4] 2> 1377646 T5706 oasc.CoreContainer.<init> New CoreContainer 519719
[junit4] 2> 1377646 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007/]
[junit4] 2> 1377647 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1377647 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1377647 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1377647 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1377648 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1377648 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1377648 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1377649 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1377649 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1377649 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1377650 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1377650 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1377650 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1377651 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1377651 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1377652 T5800 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@150dfd name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1377653 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1377655 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1377657 T5802 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16476b4 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1377658 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1377661 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1378663 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52144_sb
[junit4] 2> 1378665 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52144_sb
[junit4] 2> 1378669 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007
[junit4] 2> 1378675 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007/conf/
[junit4] 2> 1378797 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1378799 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1378799 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1379046 T5706 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 6
[junit4] 2> 1379047 T5706 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1379049 T5706 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46515
[junit4] 2> 1379049 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1379050 T5706 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1379050 T5706 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008
[junit4] 2> 1379050 T5706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/'
[junit4] 2> 1379061 T5706 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/solr.xml
[junit4] 2> 1379067 T5706 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008
[junit4] 2> 1379068 T5706 oasc.CoreContainer.<init> New CoreContainer 8026516
[junit4] 2> 1379068 T5706 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/]
[junit4] 2> 1379069 T5706 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1379069 T5706 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1379069 T5706 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1379070 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1379070 T5706 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1379070 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1379070 T5706 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1379071 T5706 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1379071 T5706 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1379072 T5706 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1379072 T5706 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1379072 T5706 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1379073 T5706 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50897/solr
[junit4] 2> 1379073 T5706 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1379074 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1379075 T5814 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@334c84 name:ZooKeeperConnection Watcher:127.0.0.1:50897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1379075 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1379077 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1379078 T5816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1913917 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1379078 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1379081 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1380083 T5706 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46515_sb
[junit4] 2> 1380109 T5706 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46515_sb
[junit4] 2> 1380147 T5706 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008
[junit4] 2> 1380154 T5706 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/conf/
[junit4] 2> 1380279 T5706 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1380280 T5706 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1380280 T5706 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1380281 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1380296 T5819 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16ce9d8 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1380296 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1380297 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1380300 T5706 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1380316 T5821 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@218c78 name:ZooKeeperConnection Watcher:127.0.0.1:50897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1380316 T5706 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1380317 T5706 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1380320 T5754 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=testOverseerQuit&action=CREATE&numShards=4&wt=javabin&version=2
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380336 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380337 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380337 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380337 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380356 T5730 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerQuit",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 1380357 T5822 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"testOverseerQuit",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"4",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 1380357 T5822 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
[junit4] 2> 1380358 T5822 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerQuit
[junit4] 2> 1380358 T5822 oascc.SolrZkClient.makePath makePath: /collections/testOverseerQuit
[junit4] 2> 1380415 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380417 T5729 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1380417 T5729 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerQuit
[junit4] 2> 1380418 T5729 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerQuit with shards [shard1, shard2, shard3, shard4]
[junit4] 2> 1380456 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1380476 T5728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5774 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5733 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5802 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5760 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380476 T5746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1380516 T5822 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection testOverseerQuit, shardNames [shard1, shard2, shard3, shard4] , replicationFactor : 2
[junit4] 2> 1380516 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard1_replica1 as part of slice shard1 of collection testOverseerQuit on 127.0.0.1:60816_sb
[junit4] 2> 1380517 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard1_replica2 as part of slice shard1 of collection testOverseerQuit on 127.0.0.1:40347_sb
[junit4] 2> 1380518 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard2_replica1 as part of slice shard2 of collection testOverseerQuit on 127.0.0.1:58213_sb
[junit4] 2> 1380518 T5753 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard1_replica1&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard1&wt=javabin&version=2
[junit4] 2> 1380519 T5721 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard1_replica2&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard1&wt=javabin&version=2
[junit4] 2> 1380520 T5721 oasc.ZkController.publish publishing core=testOverseerQuit_shard1_replica2 state=down collection=testOverseerQuit
[junit4] 2> 1380520 T5753 oasc.ZkController.publish publishing core=testOverseerQuit_shard1_replica1 state=down collection=testOverseerQuit
[junit4] 2> 1380522 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard2_replica2 as part of slice shard2 of collection testOverseerQuit on 127.0.0.1:46515_sb
[junit4] 2> 1380523 T5770 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard2_replica1&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard2&wt=javabin&version=2
[junit4] 2> 1381821 T5710 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 1299ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 1381826 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381826 T5721 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381828 T5753 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381829 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard3_replica1 as part of slice shard3 of collection testOverseerQuit on 127.0.0.1:35314_sb
[junit4] 2> 1381828 T5770 oasc.ZkController.publish publishing core=testOverseerQuit_shard2_replica1 state=down collection=testOverseerQuit
[junit4] 2> 1381830 T5770 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381830 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard3_replica2 as part of slice shard3 of collection testOverseerQuit on 127.0.0.1:41246_sb
[junit4] 2> 1381830 T5729 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1381830 T5812 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard2_replica2&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard2&wt=javabin&version=2
[junit4] 2> 1381830 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381831 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard4_replica1 as part of slice shard4 of collection testOverseerQuit on 127.0.0.1:52144_sb
[junit4] 2> 1381831 T5783 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard3_replica2&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard3&wt=javabin&version=2
[junit4] 2> 1381830 T5742 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard3_replica1&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard3&wt=javabin&version=2
[junit4] 2> 1381831 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40347/sb",
[junit4] 2> "core":"testOverseerQuit_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40347_sb",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381833 T5822 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerQuit_shard4_replica2 as part of slice shard4 of collection testOverseerQuit on 127.0.0.1:60816_sb
[junit4] 2> 1381833 T5795 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard4_replica1&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard4&wt=javabin&version=2
[junit4] 2> 1381833 T5742 oasc.ZkController.publish publishing core=testOverseerQuit_shard3_replica1 state=down collection=testOverseerQuit
[junit4] 2> 1381833 T5812 oasc.ZkController.publish publishing core=testOverseerQuit_shard2_replica2 state=down collection=testOverseerQuit
[junit4] 2> 1381833 T5783 oasc.ZkController.publish publishing core=testOverseerQuit_shard3_replica2 state=down collection=testOverseerQuit
[junit4] 2> 1381835 T5795 oasc.ZkController.publish publishing core=testOverseerQuit_shard4_replica1 state=down collection=testOverseerQuit
[junit4] 2> 1381836 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381836 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60816/sb",
[junit4] 2> "core":"testOverseerQuit_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60816_sb",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381835 T5742 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381835 T5756 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=testOverseerQuit_shard4_replica2&action=CREATE&numShards=4&collection=testOverseerQuit&shard=shard4&wt=javabin&version=2
[junit4] 2> 1381836 T5783 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381835 T5812 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381837 T5756 oasc.ZkController.publish publishing core=testOverseerQuit_shard4_replica2 state=down collection=testOverseerQuit
[junit4] 2> 1381837 T5795 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381839 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381839 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58213/sb",
[junit4] 2> "core":"testOverseerQuit_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58213_sb",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381839 T5756 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1381842 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35314/sb",
[junit4] 2> "core":"testOverseerQuit_shard3_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35314_sb",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381845 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381845 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46515/sb",
[junit4] 2> "core":"testOverseerQuit_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:46515_sb",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381847 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381847 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41246/sb",
[junit4] 2> "core":"testOverseerQuit_shard3_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41246_sb",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381851 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381852 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52144/sb",
[junit4] 2> "core":"testOverseerQuit_shard4_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52144_sb",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381854 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381854 T5729 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60816/sb",
[junit4] 2> "core":"testOverseerQuit_shard4_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60816_sb",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testOverseerQuit",
[junit4] 2> "numShards":"4",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1381856 T5728 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1381958 T5746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381959 T5733 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381959 T5774 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381959 T5802 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381958 T5788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381958 T5816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381958 T5760 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381958 T5821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381958 T5728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1381959 T5819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4] 2> 1382826 T5721 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard1_replica2
[junit4] 2> 1382828 T5721 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382828 T5721 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382828 T5721 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382829 T5753 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard1_replica1
[junit4] 2> 1382829 T5721 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382829 T5753 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382829 T5721 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-002/testOverseerQuit_shard1_replica2/'
[junit4] 2> 1382830 T5770 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard2_replica1
[junit4] 2> 1382830 T5753 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382831 T5753 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382831 T5770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382831 T5770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382831 T5753 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382832 T5770 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382832 T5753 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/testOverseerQuit_shard1_replica1/'
[junit4] 2> 1382834 T5770 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382834 T5770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/testOverseerQuit_shard2_replica1/'
[junit4] 2> 1382837 T5742 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard3_replica1
[junit4] 2> 1382837 T5783 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard3_replica2
[junit4] 2> 1382838 T5742 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382838 T5812 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard2_replica2
[junit4] 2> 1382840 T5756 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard4_replica2
[junit4] 2> 1382839 T5795 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerQuit_shard4_replica1
[junit4] 2> 1382841 T5756 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382841 T5795 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382841 T5756 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382842 T5756 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382838 T5783 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382841 T5795 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382841 T5742 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382840 T5812 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerQuit
[junit4] 2> 1382843 T5742 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382843 T5756 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382843 T5795 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382844 T5756 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-004/testOverseerQuit_shard4_replica2/'
[junit4] 2> 1382847 T5783 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382847 T5783 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382851 T5812 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1382852 T5783 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382852 T5812 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerQuit
[junit4] 2> 1382851 T5742 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382851 T5795 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382853 T5742 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-003/testOverseerQuit_shard3_replica1/'
[junit4] 2> 1382852 T5783 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-006/testOverseerQuit_shard3_replica2/'
[junit4] 2> 1382853 T5812 oascc.ZkStateReader.readConfigName path=/collections/testOverseerQuit configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1382853 T5795 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-007/testOverseerQuit_shard4_replica1/'
[junit4] 2> 1382854 T5812 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/'
[junit4] 2> 1382888 T5770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382888 T5721 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382901 T5770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1382903 T5770 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1382920 T5721 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1382923 T5753 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382923 T5795 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382924 T5770 oass.IndexSchema.readSchema [testOverseerQuit_shard2_replica1] Schema name=test
[junit4] 2> 1382924 T5721 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1382928 T5756 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382951 T5795 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1382952 T5721 oass.IndexSchema.readSchema [testOverseerQuit_shard1_replica2] Schema name=test
[junit4] 2> 1382955 T5812 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382975 T5756 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1382975 T5795 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1382981 T5756 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1382983 T5795 oass.IndexSchema.readSchema [testOverseerQuit_shard4_replica1] Schema name=test
[junit4] 2> 1382983 T5753 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1382984 T5742 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1382985 T5783 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1383000 T5753 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1383001 T5756 oass.IndexSchema.readSchema [testOverseerQuit_shard4_replica2] Schema name=test
[junit4] 2> 1383005 T5753 oass.IndexSchema.readSchema [testOverseerQuit_shard1_replica1] Schema name=test
[junit4] 2> 1383011 T5742 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1383026 T5783 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1383028 T5742 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1383032 T5783 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1383035 T5812 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1383035 T5742 oass.IndexSchema.readSchema [testOverseerQuit_shard3_replica1] Schema name=test
[junit4] 2> 1383048 T5812 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1383055 T5783 oass.IndexSchema.readSchema [testOverseerQuit_shard3_replica2] Schema name=test
[junit4] 2> 1383057 T5812 oass.IndexSchema.readSchema [testOverseerQuit_shard2_replica2] Schema name=test
[junit4] 2> 1383304 T5770 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1383310 T5770 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1383323 T5770 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1383325 T5721 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1383330 T5721 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1383330 T5753 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1383332 T5770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1383332 T5721 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1383335 T5770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1383335 T5753 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1383336 T5756 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1383338 T5753 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1383338 T5770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1383340 T5770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1383340 T5770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1383341 T5770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1383342 T5770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1383342 T5770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1383343 T5756 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1383344 T5742 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1383345 T5756 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1383347 T5721 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1383343 T5770 oasc.CoreContainer.create Creating SolrCore 'testOverseerQuit_shard2_replica1' using configuration from collection testOverseerQuit
[junit4] 2> 1383348 T5770 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1383348 T5770 oasc.SolrCore.<init> [testOverseerQuit_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/testOverseerQuit_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/testOverseerQuit_shard2_replica1/data/
[junit4] 2> 1383349 T5770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba7197
[junit4] 2> 1383359 T5742 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1383360 T5770 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-005/testOverseerQuit_shard2_replica1/data
[junit4] 2> 1383360 T5770 oasc.SolrCore.getNewIndexDir New index directory detected
[...truncated too long message...]
ckDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1414553 T5816 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data;done=false>>]
[junit4] 2> 1414553 T5816 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data
[junit4] 2> 1414553 T5816 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data/index;done=false>>]
[junit4] 2> 1414553 T5816 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001/tempDir-008/testOverseerQuit_shard2_replica2/data/index
[junit4] 2> 1414553 T5816 oascc.SolrZkClient.makePath makePath: /collections/testOverseerQuit/leaders/shard2
[junit4] 2> 1414554 T5816 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:135)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:290)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:209)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:206)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:206)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:417)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:127)
[junit4] 2> at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:122)
[junit4] 2> ... 7 more
[junit4] 2>
[junit4] 2> 1414554 T5816 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerRolesTest-69AA10045826F286-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=sr_RS_#Latn, timezone=Pacific/Fiji
[junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_05 (32-bit)/cpus=8,threads=1,free=57652400,total=226795520
[junit4] 2> NOTE: All tests run in this JVM: [HdfsBasicDistributedZkTest, TestPerFieldSimilarity, SpatialFilterTest, TestQueryUtils, BinaryUpdateRequestHandlerTest, AutoCommitTest, OverseerStatusTest, TestExtendedDismaxParser, BasicFunctionalityTest, TestModifyConfFiles, TriLevelCompositeIdRoutingTest, UpdateParamsTest, QueryElevationComponentTest, SolrInfoMBeanTest, DistribCursorPagingTest, TestCollectionAPI, CustomCollectionTest, TestRandomDVFaceting, TimeZoneUtilsTest, ShardRoutingTest, TestBadConfig, SyncSliceTest, TermVectorComponentTest, AnalyticsQueryTest, EchoParamsTest, FastVectorHighlighterTest, TestSolrQueryParserDefaultOperatorResource, ClusterStateUpdateTest, LoggingHandlerTest, ShardSplitTest, SampleTest, TestElisionMultitermQuery, TestSuggestSpellingConverter, TestCoreContainer, AnalysisAfterCoreReloadTest, SSLMigrationTest, TestUtils, TestConfigSets, CircularListTest, SolrIndexSplitterTest, FieldFacetTest, TestLFUCache, IndexSchemaTest, HdfsChaosMonkeySafeLeaderTest, PeerSyncTest, DocValuesTest, CursorPagingTest, ChaosMonkeyNothingIsSafeTest, TestManagedResourceStorage, SuggesterTest, FieldFacetExtrasTest, UUIDFieldTest, ShowFileRequestHandlerTest, BadComponentTest, TestJmxMonitoredMap, TestReloadAndDeleteDocs, TestSort, TestAnalyzeInfixSuggestions, TestManagedStopFilterFactory, DocValuesMissingTest, MigrateRouteKeyTest, CoreAdminHandlerTest, ReturnFieldsTest, SolrCoreTest, TestSystemIdResolver, TestHighFrequencyDictionaryFactory, TestRestManager, TestWriterPerf, TestGroupingSearch, TestPseudoReturnFields, TestOrdValues, URLClassifyProcessorTest, SolrTestCaseJ4Test, TermsComponentTest, TestSchemaSimilarityResource, DistributedSpellCheckComponentTest, TestSurroundQueryParser, TestPHPSerializedResponseWriter, TestMaxScoreQueryParser, TestLuceneMatchVersion, PreAnalyzedFieldTest, TestAddFieldRealTimeGet, HdfsUnloadDistributedZkTest, DirectUpdateHandlerOptimizeTest, TestCursorMarkWithoutUniqueKey, SliceStateUpdateTest, OverseerTest, MoreLikeThisHandlerTest, RecoveryZkTest, OverseerCollectionProcessorTest, UnloadDistributedZkTest, ReplicationFactorTest, TestOverriddenPrefixQueryForCustomFieldType, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, TestStressVersions, HardAutoCommitTest, TestRangeQuery, StatsComponentTest, SolrCmdDistributorTest, SoftAutoCommitTest, TestTrie, TestCSVLoader, TestUpdate, TestAtomicUpdateErrorCases, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, XsltUpdateRequestHandlerTest, LukeRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, RequestHandlersTest, TestValueSourceCache, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestCollationField, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, QueryParsingTest, JsonLoaderTest, TestComponentsName, SearchHandlerTest, TestSolrIndexConfig, CopyFieldTest, TestStressRecovery, FunctionTest, AsyncMigrateRouteKeyTest, DeleteShardTest, HttpPartitionTest, OverseerRolesTest]
[junit4] Completed on J0 in 45.30s, 1 test, 1 failure <<< FAILURES!
[...truncated 187 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 414 suites, 1702 tests, 1 failure, 36 ignored (15 assumptions)
Total time: 83 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_05 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_05) - Build #
10589 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10589/
Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseParallelGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 108 from https://127.0.0.1:59172/collection1lastClient and got 84 from https://127.0.0.1:42264/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 108 from https://127.0.0.1:59172/collection1lastClient and got 84 from https://127.0.0.1:42264/collection1
at __randomizedtesting.SeedInfo.seed([CB92848908EA47DD:4A740A917FB527E1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([CB92848908EA47DD:4A740A917FB527E1]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:317)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:659)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1141)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:236)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10843 lines...]
[junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/init-core-data-001
[junit4] 2> 72185 T199 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 72186 T199 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 72189 T199 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 72190 T199 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 72193 T200 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 72292 T199 oasc.ZkTestServer.run start zk server on port:39257
[junit4] 2> 72293 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72396 T206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@313e7bb5 name:ZooKeeperConnection Watcher:127.0.0.1:39257 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72397 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72397 T199 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 72401 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72405 T208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50ea15b1 name:ZooKeeperConnection Watcher:127.0.0.1:39257/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72405 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72405 T199 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 72409 T199 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 72411 T199 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 72412 T199 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 72417 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 72417 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 72422 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 72422 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 72426 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 72426 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 72428 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 72429 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 72431 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 72431 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 72433 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 72434 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 72436 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 72436 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 72438 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 72438 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 72440 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 72441 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 72443 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 72443 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 72446 T199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 72446 T199 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 72785 T199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 72873 T199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58949
[junit4] 2> 72901 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 72901 T199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 72901 T199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002
[junit4] 2> 72901 T199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/'
[junit4] 2> 72910 T199 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/solr.xml
[junit4] 2> 72919 T199 oasc.CoreContainer.<init> New CoreContainer 1537106
[junit4] 2> 72920 T199 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/]
[junit4] 2> 72933 T199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 72933 T199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 72934 T199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 72934 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 72934 T199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 72934 T199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 72934 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 72935 T199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 72935 T199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 72935 T199 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 72935 T199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 72936 T199 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 72936 T199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39257/solr
[junit4] 2> 72936 T199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 72937 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72940 T220 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a770032 name:ZooKeeperConnection Watcher:127.0.0.1:39257 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72940 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72944 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72948 T222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73e043ee name:ZooKeeperConnection Watcher:127.0.0.1:39257/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72949 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72950 T199 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 72953 T199 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 72956 T199 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 72959 T199 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 72964 T199 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 72967 T199 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 72969 T199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58949_
[junit4] 2> 72970 T199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58949_
[junit4] 2> 72979 T199 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 72981 T199 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 72984 T199 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:58949_
[junit4] 2> 72985 T199 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 72987 T199 oasc.Overseer.start Overseer (id=91952325766086659-127.0.0.1:58949_-n_0000000000) starting
[junit4] 2> 72989 T199 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 72997 T224 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 72998 T199 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 73002 T199 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 73005 T199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 73008 T223 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 73010 T225 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 73010 T225 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 73012 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73012 T225 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 73013 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 73014 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58949",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58949_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 73014 T223 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 73017 T223 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 73021 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73021 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74012 T225 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 74013 T225 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 74014 T225 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 74014 T225 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 74015 T225 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 74015 T225 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/collection1/'
[junit4] 2> 74016 T225 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 74016 T225 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 74016 T225 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 74045 T225 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 74053 T225 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 74055 T225 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 74060 T225 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 74165 T225 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 74171 T225 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 74172 T225 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 74184 T225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 74187 T225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 74190 T225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 74191 T225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 74192 T225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 74192 T225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 74193 T225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 74193 T225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 74193 T225 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 74194 T225 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 74194 T225 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/control/data/
[junit4] 2> 74195 T225 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20e5204
[junit4] 2> 74196 T225 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/control/data
[junit4] 2> 74196 T225 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/control/data/index/
[junit4] 2> 74196 T225 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 74197 T225 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/control/data/index
[junit4] 2> 74197 T225 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1703993354, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 74198 T225 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@61841600 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4f75151d),segFN=segments_1,generation=1}
[junit4] 2> 74199 T225 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74205 T225 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 74206 T225 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 74206 T225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 74206 T225 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 74206 T225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 74206 T225 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 74207 T225 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 74207 T225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 74207 T225 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 74208 T225 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 74209 T225 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 74209 T225 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 74209 T225 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 74210 T225 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 74210 T225 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 74210 T225 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 74211 T225 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 74213 T225 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 74215 T225 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 74215 T225 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 74216 T225 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=17.4423828125, floorSegmentMB=1.796875, forceMergeDeletesPctAllowed=22.979901078472448, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 74217 T225 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@61841600 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4f75151d),segFN=segments_1,generation=1}
[junit4] 2> 74217 T225 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74217 T225 oass.SolrIndexSearcher.<init> Opening Searcher@44fe5e43[collection1] main
[junit4] 2> 74218 T225 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 74219 T225 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 74219 T225 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 74220 T225 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 74220 T225 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 74220 T225 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 74221 T225 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 74221 T225 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 74221 T225 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 74221 T225 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 74228 T225 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 74229 T226 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44fe5e43[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 74229 T229 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58949 collection:control_collection shard:shard1
[junit4] 2> 74230 T199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 74230 T229 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 74230 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 74237 T229 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 74246 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74247 T229 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 74247 T229 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C77 name=collection1 org.apache.solr.core.SolrCore@1bdf7642 url=http://127.0.0.1:58949/collection1 node=127.0.0.1:58949_ C77_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:58949, core=collection1, node_name=127.0.0.1:58949_}
[junit4] 2> 74248 T229 C77 P58949 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58949/collection1/
[junit4] 2> 74248 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74248 T229 C77 P58949 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 74248 T229 C77 P58949 oasc.SyncStrategy.syncToMe http://127.0.0.1:58949/collection1/ has no replicas
[junit4] 2> 74249 T229 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58949/collection1/ shard1
[junit4] 2> 74249 T229 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 74251 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74253 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74256 T231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56bf7195 name:ZooKeeperConnection Watcher:127.0.0.1:39257/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74257 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74258 T199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 74258 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74261 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74262 T199 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 74363 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74363 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74409 T229 oasc.ZkController.register We are http://127.0.0.1:58949/collection1/ and leader is http://127.0.0.1:58949/collection1/
[junit4] 2> 74410 T229 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58949
[junit4] 2> 74410 T229 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 74410 T229 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 74410 T229 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 74412 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74412 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74412 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74413 T229 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74414 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74414 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:58949",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58949_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 74416 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74513 T199 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 74514 T199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 74516 T199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34402
[junit4] 2> 74518 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74518 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74521 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 74521 T199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 74521 T199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003
[junit4] 2> 74521 T199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/'
[junit4] 2> 74530 T199 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/solr.xml
[junit4] 2> 74537 T199 oasc.CoreContainer.<init> New CoreContainer 606909328
[junit4] 2> 74537 T199 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/]
[junit4] 2> 74537 T199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 74538 T199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 74538 T199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 74538 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 74538 T199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 74538 T199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 74539 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 74539 T199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 74539 T199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 74540 T199 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 74540 T199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 74540 T199 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 74540 T199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39257/solr
[junit4] 2> 74541 T199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 74541 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74543 T242 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e38df84 name:ZooKeeperConnection Watcher:127.0.0.1:39257 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74543 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74545 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74546 T244 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8b8332e name:ZooKeeperConnection Watcher:127.0.0.1:39257/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74547 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74551 T199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 75554 T199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34402_
[junit4] 2> 75555 T199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34402_
[junit4] 2> 75563 T245 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 75563 T245 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 75564 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75564 T245 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 75565 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75565 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75566 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75567 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34402",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34402_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 75567 T223 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 75567 T223 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 75572 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75674 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 75674 T244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 75674 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76565 T245 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 76565 T245 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 76566 T245 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 76566 T245 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 76567 T245 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 76567 T245 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/collection1/'
[junit4] 2> 76568 T245 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 76568 T245 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 76569 T245 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 76596 T245 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 76618 T245 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 76623 T245 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 76631 T245 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 76731 T245 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 76735 T245 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 76736 T245 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 76752 T245 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 76755 T245 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 76758 T245 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 76759 T245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 76759 T245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 76760 T245 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 76761 T245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 76761 T245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 76761 T245 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 76762 T245 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 76762 T245 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty1/
[junit4] 2> 76762 T245 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20e5204
[junit4] 2> 76763 T245 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty1
[junit4] 2> 76764 T245 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty1/index/
[junit4] 2> 76764 T245 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 76764 T245 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty1/index
[junit4] 2> 76765 T245 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1703993354, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 76766 T245 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57487887 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33812826),segFN=segments_1,generation=1}
[junit4] 2> 76766 T245 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76774 T245 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 76774 T245 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 76774 T245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 76775 T245 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 76775 T245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 76775 T245 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 76775 T245 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 76776 T245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 76776 T245 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 76777 T245 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 76777 T245 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 76778 T245 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 76778 T245 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 76781 T245 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 76782 T245 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 76783 T245 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 76783 T245 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 76786 T245 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 76787 T245 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 76788 T245 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 76789 T245 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=17.4423828125, floorSegmentMB=1.796875, forceMergeDeletesPctAllowed=22.979901078472448, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 76789 T245 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57487887 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33812826),segFN=segments_1,generation=1}
[junit4] 2> 76790 T245 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76790 T245 oass.SolrIndexSearcher.<init> Opening Searcher@2ada0303[collection1] main
[junit4] 2> 76790 T245 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 76793 T245 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 76793 T245 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 76793 T245 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 76794 T245 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 76794 T245 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 76794 T245 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 76795 T245 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 76795 T245 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 76795 T245 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 76817 T246 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ada0303[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 76820 T245 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 76821 T249 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34402 collection:collection1 shard:shard1
[junit4] 2> 76826 T199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 76826 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 76826 T249 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 76837 T249 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 76839 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76839 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76839 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76842 T249 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 76843 T249 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 76843 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C78 name=collection1 org.apache.solr.core.SolrCore@77527cd0 url=http://127.0.0.1:34402/collection1 node=127.0.0.1:34402_ C78_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:34402, core=collection1, node_name=127.0.0.1:34402_}
[junit4] 2> 76843 T249 C78 P34402 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34402/collection1/
[junit4] 2> 76844 T249 C78 P34402 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 76845 T249 C78 P34402 oasc.SyncStrategy.syncToMe http://127.0.0.1:34402/collection1/ has no replicas
[junit4] 2> 76845 T249 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34402/collection1/ shard1
[junit4] 2> 76846 T249 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 76847 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76854 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76858 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76968 T244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76968 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76968 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77014 T249 oasc.ZkController.register We are http://127.0.0.1:34402/collection1/ and leader is http://127.0.0.1:34402/collection1/
[junit4] 2> 77015 T249 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34402
[junit4] 2> 77015 T249 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77015 T249 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 77015 T249 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77017 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77017 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77018 T249 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77018 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77021 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77022 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:34402",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34402_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 77026 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77130 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77130 T244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77130 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77245 T199 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 77246 T199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 77249 T199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34344
[junit4] 2> 77249 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 77250 T199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 77250 T199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004
[junit4] 2> 77250 T199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/'
[junit4] 2> 77280 T199 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/solr.xml
[junit4] 2> 77306 T199 oasc.CoreContainer.<init> New CoreContainer 1642559176
[junit4] 2> 77307 T199 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/]
[junit4] 2> 77308 T199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 77308 T199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 77308 T199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 77309 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77309 T199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77309 T199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77309 T199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77310 T199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77310 T199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77311 T199 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77311 T199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77312 T199 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77312 T199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39257/solr
[junit4] 2> 77312 T199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77313 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77316 T260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62b69e6b name:ZooKeeperConnection Watcher:127.0.0.1:39257 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77317 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77321 T199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77327 T262 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d3b0deb name:ZooKeeperConnection Watcher:127.0.0.1:39257/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77327 T199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77345 T199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 78348 T199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34344_
[junit4] 2> 78349 T199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34344_
[junit4] 2> 78364 T263 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 78364 T263 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78366 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78366 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78367 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78367 T263 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78368 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78369 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34344",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34344_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78369 T223 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 78369 T223 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 78376 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78478 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78478 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78479 T244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78479 T262 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79367 T263 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79368 T263 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79368 T263 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79369 T263 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79369 T263 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79369 T263 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/collection1/'
[junit4] 2> 79370 T263 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 79371 T263 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 79371 T263 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 79406 T263 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 79418 T263 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79420 T263 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79435 T263 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79559 T263 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 79565 T263 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79567 T263 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79576 T263 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 79578 T263 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 79581 T263 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 79581 T263 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 79582 T263 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 79582 T263 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 79583 T263 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 79583 T263 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 79583 T263 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 79583 T263 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79583 T263 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty2/
[junit4] 2> 79584 T263 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20e5204
[junit4] 2> 79584 T263 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty2
[junit4] 2> 79584 T263 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty2/index/
[junit4] 2> 79584 T263 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 79585 T263 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-001/jetty2/index
[junit4] 2> 79585 T263 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1703993354, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 79586 T263 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5490ba5a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d16f709),segFN=segments_1,generation=1}
[junit4] 2> 79586 T263 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79593 T263 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 79593 T263 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 79594 T263 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 79594 T263 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 79594 T263 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 79594 T263 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 79594 T263 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 79595 T263 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 79595 T263 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 79595 T263 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 79596 T263 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 79596 T263 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 79596 T263 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 79597 T263 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 79597 T263 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 79598 T263 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 79598 T263 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 79601 T263 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 79608 T263 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 79609 T263 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 79612 T263 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=17.4423828125, floorSegmentMB=1.796875, forceMergeDeletesPctAllowed=22.979901078472448, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 79613 T263 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5490ba5a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d16f709),segFN=segments_1,generation=1}
[junit4] 2> 79613 T263 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79614 T263 oass.SolrIndexSearcher.<init> Opening Searcher@72904bd7[collection1] main
[junit4] 2> 79614 T263 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79615 T263 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79615 T263 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 79616 T263 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 79617 T263 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 79617 T263 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 79618 T263 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 79618 T263 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 79618 T263 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 79619 T263 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 79638 T264 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72904bd7[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 79642 T263 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 79644 T267 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34344 collection:collection1 shard:shard1
[junit4] 2> 79645 T199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 79645 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 79654 T267 oasc.ZkController.register We are http://127.0.0.1:34344/collection1/ and leader is http://127.0.0.1:34402/collection1/
[junit4] 2> 79655 T267 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34344
[junit4] 2> 79655 T267 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 79655 T267 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C79 name=collection1 org.apache.solr.core.SolrCore@54f04944 url=http://127.0.0.1:34344/collection1 node=127.0.0.1:34344_ C79_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:34344, core=collection1, node_name=127.0.0.1:34344_}
[junit4] 2> 79693 T268 C79 P34344 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 79693 T267 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79694 T268 C79 P34344 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 79694 T268 C79 P34344 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is http://127.0.0.1:34402/collection1/ and I am http://127.0.0.1:34344/collection1/
[junit4] 2> 79695 T268 C79 P34344 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 79695 T268 C79 P34344 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 79697 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79697 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79697 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79701 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79702 T223 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"http://127.0.0.1:34344",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34344_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 79706 T222 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79737 T268 C79 P34344 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to http://127.0.0.1:34402; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A34344_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> 79808 T244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79809 T262 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79809 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79809 T231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80119 T238 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 80121 T238 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80132 T238 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 80134 T238 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34344_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"state":"recovering","base_url":"http://127.0.0.1:34344","core":"collection1","node_name":"127.0.0.1:34344_"}
[junit4] 2> 80136 T238 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 80137 T238 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:34344_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 80158 T199 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 80159 T199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 80164 T199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53829
[junit4] 2> 80165 T199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 80166 T199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 80166 T199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-005
[junit4] 2> 80166 T199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-005/'
[junit4] 2> 80197 T199 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-005/solr.xml
[junit4] 2> 80216 T199 oasc.CoreContainer.<init> New CoreContainer 955620845
[junit4] 2> 80217 T199 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-CB92848908EA47DD-001/tempDir-005/]
[junit4] 2> 80218 T199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 80219 T199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 80219 T199 oashc.HttpShardHandlerFactory.getPara
[...truncated too long message...]
est-CB92848908EA47DD-001/tempDir-001/jetty9/index.20140618105038506 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty9/index.20140618105038506;done=false>>]
[junit4] 2> 380967 T1800 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty9/index.20140618105038506
[junit4] 2> 380967 T1800 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty9 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty9;done=false>>]
[junit4] 2> 380967 T1800 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty9
[junit4] 2> 380967 T1800 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 380968 T1800 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 381245 T1823 C1177 P47588 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 381245 T1823 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 381245 T1823 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1ff42243
[junit4] 2> 381258 T1823 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=16,cumulative_deletesById=4,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=9549,transaction_logs_total_number=3}
[junit4] 2> 381259 T1823 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 381259 T1823 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 381259 T1823 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 381265 T1823 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 381265 T1823 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 381266 T1823 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12/index.20140618105041498 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12/index.20140618105041498;done=false>>]
[junit4] 2> 381266 T1823 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12/index.20140618105041498
[junit4] 2> 381266 T1823 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12;done=false>>]
[junit4] 2> 381266 T1823 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001/tempDir-001/jetty12
[junit4] 2> 381266 T1823 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 381267 T1823 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-CB92848908EA47DD-001
[junit4] 2> NOTE: test params are: codec=Lucene49: {rnd_b=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=FSTPulsing41), a_i=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=DefaultSimilarity, locale=zh, timezone=Africa/Tripoli
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_05 (64-bit)/cpus=8,threads=1,free=147042224,total=406847488
[junit4] 2> NOTE: All tests run in this JVM: [TestSimpleQParserPlugin, TestSolrQueryParserResource, DisMaxRequestHandlerTest, AnalysisErrorHandlingTest, TestSolrDeletionPolicy2, TestIndexSearcher, OverseerRolesTest, TestXIncludeConfig, TestCloudInspectUtil, TestClassNameShortening, TestComplexPhraseQParserPlugin, TestNoOpRegenerator, CoreAdminRequestStatusTest, TestArbitraryIndexDir, JSONWriterTest, DocumentAnalysisRequestHandlerTest, BasicDistributedZk2Test, TestFoldingMultitermQuery, IndexBasedSpellCheckerTest, MultiThreadedOCPTest, TestZkChroot, TestRecovery, TestRandomFaceting, TestShortCircuitedRequests, TestHashPartitioner, TestCodecSupport, SolrRequestParserTest, TestFastOutputStream, TestRecoveryHdfs, RequestHandlersTest, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J1 in 81.43s, 1 test, 1 failure <<< FAILURES!
[...truncated 1009 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 414 suites, 1702 tests, 1 error, 1 failure, 36 ignored (15 assumptions)
Total time: 61 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
10588 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10588/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
2 tests failed.
REGRESSION: org.apache.solr.cloud.DistribCursorPagingTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:47992/collection1, http://127.0.0.1:55207/collection1, http://127.0.0.1:32927/collection1, http://127.0.0.1:45704/collection1]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:47992/collection1, http://127.0.0.1:55207/collection1, http://127.0.0.1:32927/collection1, http://127.0.0.1:45704/collection1]
at __randomizedtesting.SeedInfo.seed([42A96F902BF55183:C34FE1885CAA31BF]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:319)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:659)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.DistribCursorPagingTest.assertFullWalkNoDups(DistribCursorPagingTest.java:711)
at org.apache.solr.cloud.DistribCursorPagingTest.doRandomSortsOnLargeIndex(DistribCursorPagingTest.java:586)
at org.apache.solr.cloud.DistribCursorPagingTest.doTest(DistribCursorPagingTest.java:90)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:619)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:853)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55207/collection1, http://127.0.0.1:32927/collection1]
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
... 46 more
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
at __randomizedtesting.SeedInfo.seed([42A96F902BF55183:46A1E0633950BEA2]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:134)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:619)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:853)
Build Log:
[...truncated 10785 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerTest-42A96F902BF55183-001/init-core-data-001
[junit4] 2> 206119 T650 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 206119 T650 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 206119 T650 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 206122 T650 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
[junit4] 2> 206122 T650 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 206123 T651 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 206223 T650 oasc.ZkTestServer.run start zk server on port:47651
[junit4] 2> 206224 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 207115 T657 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34d5f876 name:ZooKeeperConnection Watcher:127.0.0.1:47651 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 207115 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 207118 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 207120 T659 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bbca8dd6 name:ZooKeeperConnection Watcher:127.0.0.1:47651 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 207120 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 207121 T650 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 207124 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 207125 T661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d9701103 name:ZooKeeperConnection Watcher:127.0.0.1:47651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 207125 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 207126 T650 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 207128 T650 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 207130 T650 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 207132 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 207135 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 207137 T663 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8f5ad5e6 name:ZooKeeperConnection Watcher:127.0.0.1:47651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 207138 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 207139 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 207141 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 207144 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 207146 T665 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f0edf27 name:ZooKeeperConnection Watcher:127.0.0.1:47651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 207146 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 207146 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 207149 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 207152 T650 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47651_solr
[junit4] 2> 207153 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 207155 T650 oasc.Overseer.start Overseer (id=91951428770267140-127.0.0.1:47651_solr-n_0000000000) starting
[junit4] 2> 207157 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 207160 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 207164 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 207166 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 207169 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 207172 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 207179 T667 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 207180 T666 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 208181 T665 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208182 T666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208183 T666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 208184 T666 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 208184 T666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 208186 T665 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208187 T663 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208187 T661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208681 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 208688 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 208692 T665 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208693 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208693 T666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208696 T665 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208696 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 208698 T666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 208698 T666 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 208700 T665 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208802 T661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208802 T663 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 209181 T667 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4] 2> 209855 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 209860 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 209862 T669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3acb1732 name:ZooKeeperConnection Watcher:127.0.0.1:47651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 209862 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 209863 T666 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:47651_solr
[junit4] 2> 209864 T670 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 209866 T650 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47651_solr
[junit4] 2> 209866 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 209868 T650 oasc.Overseer.start Overseer (id=91951428770267141-127.0.0.1:47651_solr-n_0000000001) starting
[junit4] 2> 209879 T672 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 209882 T671 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 209890 T671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 209891 T671 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 209892 T671 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 209895 T669 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 209895 T663 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 209896 T661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 209950 T669 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 209950 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 209952 T671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 209960 T650 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/91951428770267139-node1_core1-n_0000000002
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:127)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
[junit4] 2> at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:619)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at java.lang.Thread.run(Thread.java:853)
[junit4] 2>
[junit4] 2> 209963 T663 oasc.LeaderElector.checkIfIamLeader WARN Our node is no longer in line to be leader
[junit4] 2> 209969 T663 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 209973 T650 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:47651 47651
[junit4] 2> 210515 T651 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:47651 47651
[junit4] 2> 210516 T650 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 210516 T671 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:47651_solr
[junit4] 2> 210517 T673 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=42A96F902BF55183 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_CH -Dtests.timezone=America/Edmonton -Dtests.file.encoding=UTF-8
[junit4] ERROR 4.41s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] > at __randomizedtesting.SeedInfo.seed([42A96F902BF55183:46A1E0633950BEA2]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:134)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
[junit4] > at java.lang.Thread.run(Thread.java:853)
[junit4] 2> 210533 T650 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 210533 T650 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 210534 T674 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 210634 T650 oasc.ZkTestServer.run start zk server on port:36349
[junit4] 2> 210635 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 210639 T680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9955a19a name:ZooKeeperConnection Watcher:127.0.0.1:36349/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 210639 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 210640 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 210642 T682 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64e2cdfe name:ZooKeeperConnection Watcher:127.0.0.1:36349 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 210642 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 210643 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 210645 T684 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fa31430 name:ZooKeeperConnection Watcher:127.0.0.1:36349 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 210645 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 210645 T650 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 210647 T650 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 210649 T650 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 210651 T650 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 210652 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 210654 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 210655 T686 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e8a8fc5d name:ZooKeeperConnection Watcher:127.0.0.1:36349/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 210655 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 210656 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 210658 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 210660 T650 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36349_solr
[junit4] 2> 210661 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 210662 T650 oasc.Overseer.start Overseer (id=91951429059346435-127.0.0.1:36349_solr-n_0000000000) starting
[junit4] 2> 210664 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 210666 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 210668 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 210670 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 210676 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 210679 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 210685 T688 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 210686 T687 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 210687 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 210688 T687 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 210689 T687 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 210689 T687 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 210691 T686 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 210691 T680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 210738 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 210740 T686 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 210742 T687 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 210743 T687 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 210743 T687 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 210743 T687 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 210745 T686 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 210848 T680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 210894 T650 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:36349 36349
[junit4] 2> 211470 T663 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 211583 T674 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:36349 36349
[junit4] 2> 211584 T650 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 2> 211584 T687 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:36349_solr
[junit4] 2> 211585 T689 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 211589 T650 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 211590 T650 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 211592 T690 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 211691 T650 oasc.ZkTestServer.run start zk server on port:45564
[junit4] 2> 211693 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211695 T696 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fcf7b6f5 name:ZooKeeperConnection Watcher:127.0.0.1:45564 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211695 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211697 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211698 T698 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dda6d7a5 name:ZooKeeperConnection Watcher:127.0.0.1:45564 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211698 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211699 T650 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 211702 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211703 T700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9b03ed84 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211703 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211704 T650 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 211706 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211707 T702 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21e2ab23 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211707 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211708 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 211710 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 211714 T650 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45564_solr
[junit4] 2> 211714 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 211716 T650 oasc.Overseer.start Overseer (id=91951429128749059-127.0.0.1:45564_solr-n_0000000000) starting
[junit4] 2> 211718 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 211720 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 211722 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 211727 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 211730 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 211732 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 211736 T704 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 211737 T703 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 211738 T650 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 211740 T650 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 211741 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211743 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211744 T706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eee9fdce name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211744 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211745 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211746 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 211749 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211750 T708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49d05ffe name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211750 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211751 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211752 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 211754 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211756 T710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3033c044 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211756 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211757 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211759 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 211761 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211762 T712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e5066769 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211762 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211763 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211765 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 211768 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211769 T714 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34946466 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211769 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211770 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211772 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 211775 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211777 T716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20211533 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211777 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211778 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211780 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 211783 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211784 T718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6895772a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211784 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211786 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211788 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 211791 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211792 T720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ca958d3c name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211793 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211794 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211796 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 211799 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211801 T722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a71204a4 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211801 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211802 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211804 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 211808 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211810 T724 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a1b18c2 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211810 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211811 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211813 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 211817 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211819 T726 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2a4f54f name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211819 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211820 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211823 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 211828 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211829 T728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c8668fa2 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211829 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211830 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211832 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 211835 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211836 T730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25770685 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211837 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211837 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211839 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 211844 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211846 T732 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43368bfb name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211846 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211847 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211848 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 211853 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211855 T734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d3a1cbed name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211856 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211857 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211858 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 211863 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211864 T736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd780b1c name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211864 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211865 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211867 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 211873 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211874 T738 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e545a97 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211874 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211875 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211877 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 211880 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211882 T740 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9b844edd name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211882 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211885 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211886 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 211890 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211892 T742 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@da5cdfce name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211892 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211894 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211897 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 211906 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211909 T744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3419b486 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211909 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211911 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211913 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 211920 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211921 T746 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fef6a01a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211922 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211923 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211925 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 211929 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211932 T748 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30376da3 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211932 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211934 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211935 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 211942 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211945 T750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d4ccabe6 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211945 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211946 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211948 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 211955 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211957 T752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cc72ced7 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211957 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211959 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211960 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 211970 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211972 T754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bb16953a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211973 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211975 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211978 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 211985 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211986 T756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@667fb124 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211986 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211987 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211989 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 211995 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211996 T758 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c3262d4 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211997 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211997 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211999 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 212006 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212007 T760 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@201cb6b7 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212007 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212008 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212010 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 212016 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212017 T762 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@965a2e52 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212018 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212018 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212020 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 212027 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212029 T764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@244ca779 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212029 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212030 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212032 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 212040 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212041 T766 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ebfe6ac9 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212041 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212042 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212044 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 212052 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212053 T768 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4623f5dd name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212053 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212054 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212056 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 212064 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212067 T770 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2dc6dbf name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212067 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212068 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212072 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 212085 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212087 T772 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e0c7033 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212087 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212088 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212090 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node33
[junit4] 2> 212099 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212101 T774 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e3db8837 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212101 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212102 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212104 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node34
[junit4] 2> 212110 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212111 T776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@600ffc7b name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212111 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212112 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212114 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node35
[junit4] 2> 212121 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212122 T778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ec349542 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212122 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212123 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212124 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node36
[junit4] 2> 212131 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212132 T780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@be84b0f1 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212132 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212133 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212135 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node37
[junit4] 2> 212143 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212145 T782 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2f72f43 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212145 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212147 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212149 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node38
[junit4] 2> 212160 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212161 T784 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cb67b570 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212161 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212162 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212164 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node39
[junit4] 2> 212177 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212179 T786 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d0ffba43 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212179 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212180 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212182 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node40
[junit4] 2> 212190 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212192 T788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9391b051 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212192 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212193 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212194 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node41
[junit4] 2> 212206 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212207 T790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58b766ec name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212207 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212208 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212210 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node42
[junit4] 2> 212218 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212220 T792 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8a405e39 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212220 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212221 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212223 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node43
[junit4] 2> 212230 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212231 T794 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f63ea062 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212232 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212232 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212234 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node44
[junit4] 2> 212243 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212245 T796 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72f1eafb name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212245 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212246 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212247 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node45
[junit4] 2> 212256 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212257 T798 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79ae214e name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212257 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212258 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212260 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node46
[junit4] 2> 212268 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212270 T800 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33387b1a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212270 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212271 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212272 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node47
[junit4] 2> 212281 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212282 T802 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e861ab8a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212282 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212283 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212285 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node48
[junit4] 2> 212302 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212303 T804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2b0f26f name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212303 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212304 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212306 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node49
[junit4] 2> 212319 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212320 T806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2da9b164 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212320 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212321 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212323 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node50
[junit4] 2> 212333 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212334 T808 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7588d11b name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212334 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212335 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212337 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node51
[junit4] 2> 212353 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212354 T810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@491d2a9c name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212355 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212356 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212357 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node52
[junit4] 2> 212368 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212370 T812 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@220d0b78 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212371 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212372 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212374 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node53
[junit4] 2> 212384 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212386 T814 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b64ca055 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212386 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212387 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212389 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node54
[junit4] 2> 212400 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212401 T816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@126da85a name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212401 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212402 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212403 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node55
[junit4] 2> 212414 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212416 T818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e118600d name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212416 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212417 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212419 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node56
[junit4] 2> 212432 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212433 T820 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b9d1b022 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212433 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212434 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212436 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node57
[junit4] 2> 212450 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212451 T822 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c3f6a2b8 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212451 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212452 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212455 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node58
[junit4] 2> 212469 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212470 T824 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c49779a2 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212470 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212471 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212473 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node59
[junit4] 2> 212487 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212489 T826 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9070f931 name:ZooKeeperConnection Watcher:127.0.0.1:45564/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212490 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212491 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 212494 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/node60
[junit4] 2> 212512 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212514 T828 oascc.ConnectionManager.proce
[...truncated too long message...]
cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.DistribCursorPagingTest-42A96F902BF55183-001
[junit4] 2> NOTE: test params are: codec=Lucene49: {double_last=PostingsFormat(name=FSTOrdPulsing41), uuid=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), currency_last____currency=PostingsFormat(name=Lucene41VarGapFixedInterval), currency_last____amount_raw=PostingsFormat(name=Lucene41VarGapFixedInterval), long_last=PostingsFormat(name=Lucene41VarGapFixedInterval), long=PostingsFormat(name=SimpleText), collation_en_primary_first=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), long_first=PostingsFormat(name=Lucene41VarGapFixedInterval), enum_last=PostingsFormat(name=FSTOrdPulsing41), enum_first=PostingsFormat(name=FSTOrdPulsing41), int=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), double=PostingsFormat(name=Lucene41VarGapFixedInterval), uuid_first=PostingsFormat(name=FSTOrdPulsing41), float_last=PostingsFormat(name=Lucene41VarGapFixedInterval), int_last=PostingsFormat(name=FSTOrdPulsing41), bool_last=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), float_first=PostingsFormat(name=Lucene41VarGapFixedInterval), date=PostingsFormat(name=FSTOrdPulsing41), bool=PostingsFormat(name=FSTOrdPulsing41), icu_collation_en_primary_last=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), double_first=PostingsFormat(name=FSTOrdPulsing41), float=PostingsFormat(name=SimpleText), uuid_last=PostingsFormat(name=FSTOrdPulsing41), enum=PostingsFormat(name=Lucene41VarGapFixedInterval), int_first=PostingsFormat(name=FSTOrdPulsing41), date_last=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), icu_collation_en_primary_first=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), str_first=PostingsFormat(name=SimpleText), bool_first=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), str_last=PostingsFormat(name=FSTOrdPulsing41), currency____amount_raw=PostingsFormat(name=SimpleText), currency_first____currency=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), currency_first____amount_raw=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), collation_en_primary_last=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), _version_=PostingsFormat(name=SimpleText), id=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), str=PostingsFormat(name=Lucene41VarGapFixedInterval), date_first=Pulsing41(freqCutoff=3 minBlockSize=3 maxBlockSize=28), currency____currency=PostingsFormat(name=FSTOrdPulsing41)}, docValues:{str_dv_first=DocValuesFormat(name=Lucene49), str_dv=DocValuesFormat(name=Memory), double_dv_first=DocValuesFormat(name=Lucene49), date_dv_first=DocValuesFormat(name=Memory), bin_dv_first=DocValuesFormat(name=SimpleText), uuid_dv_first=DocValuesFormat(name=SimpleText), double_dv=DocValuesFormat(name=Memory), icu_collation_en_primary_dv_last=DocValuesFormat(name=Asserting), float_dv=DocValuesFormat(name=Lucene49), icu_collation_en_primary_dv=DocValuesFormat(name=SimpleText), collation_en_primary_dv=DocValuesFormat(name=SimpleText), long_dv=DocValuesFormat(name=Lucene49), icu_collation_en_primary_dv_first=DocValuesFormat(name=Memory), collation_en_primary_dv_last=DocValuesFormat(name=Memory), bin_dv=DocValuesFormat(name=Asserting), int_dv_last=DocValuesFormat(name=SimpleText), double_dv_last=DocValuesFormat(name=Lucene49), bin_dv_last=DocValuesFormat(name=SimpleText), date_dv_last=DocValuesFormat(name=Asserting), int_dv_first=DocValuesFormat(name=SimpleText), int_dv=DocValuesFormat(name=Asserting), long_dv_first=DocValuesFormat(name=Asserting), uuid_dv=DocValuesFormat(name=Asserting), uuid_dv_last=DocValuesFormat(name=SimpleText), date_dv=DocValuesFormat(name=SimpleText), float_dv_first=DocValuesFormat(name=Asserting), float_dv_last=DocValuesFormat(name=Memory), str_dv_last=DocValuesFormat(name=Lucene49), collation_en_primary_dv_first=DocValuesFormat(name=Memory), long_dv_last=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ar_SA, timezone=America/Halifax
[junit4] 2> NOTE: Linux 3.8.0-41-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=63653096,total=300941312
[junit4] 2> NOTE: All tests run in this JVM: [ParsingFieldUpdateProcessorsTest, TestSerializedLuceneMatchVersion, TestBadConfig, SuggesterFSTTest, NoFacetTest, HdfsBasicDistributedZk2Test, TestDynamicFieldResource, ResponseLogComponentTest, QueryParsingTest, TestNonDefinedSimilarityFactory, TestSolrDeletionPolicy1, TestRealTimeGet, SolrXmlInZkTest, TestFaceting, CollectionsAPIAsyncDistributedZkTest, DistributedQueryElevationComponentTest, TestReplicationHandlerBackup, MoreLikeThisHandlerTest, DistribDocExpirationUpdateProcessorTest, OverseerTest, TestStressVersions, CoreAdminCreateDiscoverTest, TestTrie, CopyFieldTest, CoreAdminRequestStatusTest, SpellingQueryConverterTest, TestDocSet, ReturnFieldsTest, TestDocBasedVersionConstraints, SOLR749Test, TestReRankQParserPlugin, TestFieldTypeResource, TestElisionMultitermQuery, HdfsSyncSliceTest, JsonLoaderTest, QueryResultKeyTest, OverseerCollectionProcessorTest, SolrCoreTest, TestReplicationHandler, CursorMarkTest, TestExtendedDismaxParser, TestBM25SimilarityFactory, AnalysisErrorHandlingTest, TestSearcherReuse, SoftAutoCommitTest, DistributedSpellCheckComponentTest, LukeRequestHandlerTest, ModifyConfFileTest, TestConfig, RemoteQueryErrorTest, TestOrdValues, ScriptEngineTest, TestDefaultSearchFieldResource, FieldMutatingUpdateProcessorTest, QueryElevationComponentTest, EnumFieldTest, AbstractAnalyticsStatsTest, TestValueSourceCache, BasicDistributedZkTest, HdfsBasicDistributedZkTest, URLClassifyProcessorTest, TestSchemaSimilarityResource, SyncSliceTest, AnalyticsQueryTest, ZkCLITest, TestDistributedMissingSort, TestSort, ShowFileRequestHandlerTest, TestManagedStopFilterFactory, MultiThreadedOCPTest, TestCollationFieldDocValues, FieldFacetTest, TestRemoteStreaming, DocValuesTest, DocumentBuilderTest, SolrInfoMBeanTest, TestSolrXmlPersistence, TestMergePolicyConfig, SolrTestCaseJ4Test, BadComponentTest, TestRandomDVFaceting, TestFunctionQuery, RegexBoostProcessorTest, ShardRoutingTest, ClusterStateTest, SchemaVersionSpecificBehaviorTest, TestDynamicFieldCollectionResource, StatelessScriptUpdateProcessorFactoryTest, TestGroupingSearch, TestSolrXml, CoreAdminHandlerTest, SearchHandlerTest, DirectUpdateHandlerOptimizeTest, OverseerRolesTest, AutoCommitTest, RAMDirectoryFactoryTest, ChaosMonkeyNothingIsSafeTest, TestSearchPerf, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, ShardRoutingCustomTest, SliceStateUpdateTest, TestDistributedGrouping, TermVectorComponentDistributedTest, ZkControllerTest, TestJoin, PeerSyncTest, BadIndexSchemaTest, TestFiltering, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, QueryEqualityTest, SortByFunctionTest, DebugComponentTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestQueryUtils, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, RequestHandlersTest, TestIndexingPerformance, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, IndexSchemaTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, CoreContainerCoreInitFailuresTest, HighlighterConfigTest, TestSolrIndexConfig, SolrIndexConfigTest, MultiTermTest, SampleTest, TestBinaryField, MinimalSchemaTest, ExternalFileFieldSortTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestDefaultSimilarityFactory, TestJmxMonitoredMap, TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, DateMathParserTest, TestSolrXMLSerializer, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, TestUtils, SliceStateTest, UUIDFieldTest, CursorPagingTest, AsyncMigrateRouteKeyTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistribCursorPagingTest]
[junit4] Completed on J0 in 52.52s, 1 test, 1 error <<< FAILURES!
[...truncated 211 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:460: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 414 suites, 1702 tests, 2 errors, 44 ignored (15 assumptions)
Total time: 74 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any