You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/01/07 05:58:30 UTC
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b02) - Build
# 8893 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8893/
Java: 64bit/jdk1.7.0_60-ea-b02 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([E583EF83C3C4E90F:6465619BB49B8933]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.junit.Assert.assertFalse(Assert.java:79)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:390)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:203)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:849)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10131 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> 107453 T286 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 107457 T286 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1389069649769
[junit4] 2> 107458 T286 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 107458 T287 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 107558 T286 oasc.ZkTestServer.run start zk server on port:60536
[junit4] 2> 107559 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107562 T293 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b8235e9 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107562 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107563 T286 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 107568 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107569 T295 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40c17724 name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107569 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107570 T286 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 107573 T286 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 107577 T286 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 107579 T286 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 107584 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 107585 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 107593 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 107594 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 107599 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 107599 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 107603 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 107604 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 107608 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 107608 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 107613 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 107613 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 107618 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 107618 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 107622 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 107623 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 107626 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 107627 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 107631 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 107632 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 107636 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 107636 T286 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> Using old style solr.xml
[junit4] 2> 107643 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107645 T297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@251f38a7 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107645 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107645 T286 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 107651 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107653 T299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35f53e26 name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107653 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107653 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 107654 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 107660 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 107661 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
[junit4] 2> 107665 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 107665 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 107669 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 107670 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
[junit4] 2> 107673 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 107674 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
[junit4] 2> 107678 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 107679 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
[junit4] 2> 107683 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 107683 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 107687 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 107687 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 107692 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 107692 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 107696 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 107696 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
[junit4] 2> 107700 T286 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 107701 T286 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
[junit4] 2> 107781 T286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 107783 T286 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40421
[junit4] 2> 107784 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 107785 T286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 107785 T286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018
[junit4] 2> 107785 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/'
[junit4] 2> 107794 T286 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solr.xml
[junit4] 2> 107823 T286 oasc.CoreContainer.<init> New CoreContainer 480786465
[junit4] 2> 107823 T286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/]
[junit4] 2> 107824 T286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 107824 T286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 107824 T286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 107825 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 107825 T286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 107825 T286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 107825 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 107826 T286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 107826 T286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 107829 T286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 107829 T286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 107830 T286 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 107830 T286 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60536/solr
[junit4] 2> 107830 T286 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 107831 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107833 T310 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@658c0944 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107833 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107836 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107838 T312 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b51a752 name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107838 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107840 T286 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 107845 T286 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 107849 T286 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 107851 T286 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40421_
[junit4] 2> 107852 T286 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40421_
[junit4] 2> 107856 T286 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 107859 T286 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 107865 T286 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 107868 T286 oasc.Overseer.start Overseer (id=91034068567392261-127.0.0.1:40421_-n_0000000000) starting
[junit4] 2> 107873 T286 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 107877 T314 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 107878 T286 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 107883 T286 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 107886 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 107890 T313 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 107892 T286 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 107892 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 107896 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107898 T317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ea7f9d8 name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107898 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107900 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 107904 T286 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 107977 T286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 107979 T286 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37596
[junit4] 2> 107980 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 107980 T286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 107980 T286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216
[junit4] 2> 107981 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/'
[junit4] 2> 107990 T286 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solr.xml
[junit4] 2> 108017 T286 oasc.CoreContainer.<init> New CoreContainer 835603916
[junit4] 2> 108018 T286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/]
[junit4] 2> 108018 T286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 108019 T286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 108019 T286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 108019 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 108019 T286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 108020 T286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 108020 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 108020 T286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 108020 T286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 108023 T286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 108023 T286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 108024 T286 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 108024 T286 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60536/solr
[junit4] 2> 108024 T286 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 108025 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 108027 T328 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f55a59f name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 108027 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 108030 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 108032 T330 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6dac3e37 name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 108033 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 108038 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 109043 T286 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37596_
[junit4] 2> 109045 T286 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37596_
[junit4] 2> 109049 T317 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 109049 T312 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 109049 T330 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 109056 T286 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 109056 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 109127 T286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 109129 T286 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40274
[junit4] 2> 109130 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 109131 T286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 109131 T286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389069651368
[junit4] 2> 109131 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389069651368/'
[junit4] 2> 109140 T286 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389069651368/solr.xml
[junit4] 2> 109171 T286 oasc.CoreContainer.<init> New CoreContainer 378232545
[junit4] 2> 109171 T286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389069651368/]
[junit4] 2> 109172 T286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 109172 T286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 109172 T286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 109173 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 109173 T286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 109173 T286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 109173 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 109174 T286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 109174 T286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 109177 T286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 109177 T286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 109177 T286 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 109178 T286 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60536/solr
[junit4] 2> 109178 T286 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 109179 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 109181 T342 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18021157 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 109181 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 109184 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 109188 T344 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ce7c4eb name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 109188 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 109193 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 110197 T286 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40274_
[junit4] 2> 110199 T286 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40274_
[junit4] 2> 110203 T312 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 110203 T317 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 110203 T330 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 110204 T344 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 110210 T286 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 110210 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 110278 T286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 110280 T286 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40862
[junit4] 2> 110280 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 110281 T286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 110281 T286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521
[junit4] 2> 110281 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/'
[junit4] 2> 110290 T286 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solr.xml
[junit4] 2> 110317 T286 oasc.CoreContainer.<init> New CoreContainer 692199512
[junit4] 2> 110318 T286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/]
[junit4] 2> 110318 T286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 110319 T286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 110319 T286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 110319 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 110319 T286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 110320 T286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 110320 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 110320 T286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 110320 T286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 110323 T286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 110323 T286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 110324 T286 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 110324 T286 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60536/solr
[junit4] 2> 110324 T286 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 110325 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 110328 T356 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34598691 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 110328 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 110333 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 110336 T358 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a9ca30f name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 110336 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 110342 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 111346 T286 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40862_
[junit4] 2> 111348 T286 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40862_
[junit4] 2> 111351 T344 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 111352 T330 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 111352 T317 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 111352 T312 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 111353 T358 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 111361 T286 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 111361 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 111458 T286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 111461 T286 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52002
[junit4] 2> 111462 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 111462 T286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 111462 T286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673
[junit4] 2> 111462 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/'
[junit4] 2> 111471 T286 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solr.xml
[junit4] 2> 111498 T286 oasc.CoreContainer.<init> New CoreContainer 1429450000
[junit4] 2> 111498 T286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/]
[junit4] 2> 111499 T286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 111499 T286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 111499 T286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 111500 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 111500 T286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 111500 T286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 111500 T286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 111500 T286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 111501 T286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 111503 T286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 111504 T286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 111504 T286 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 111504 T286 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60536/solr
[junit4] 2> 111505 T286 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 111505 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111507 T370 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33a72123 name:ZooKeeperConnection Watcher:127.0.0.1:60536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111508 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111511 T286 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 111513 T372 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7158ae name:ZooKeeperConnection Watcher:127.0.0.1:60536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 111513 T286 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 111521 T286 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 112527 T286 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52002_
[junit4] 2> 112529 T286 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52002_
[junit4] 2> 112533 T317 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112533 T330 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112533 T344 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112533 T312 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112533 T358 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112533 T372 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 112541 T286 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 112541 T286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 112551 T324 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&router.field=myOwnField&collection.configName=conf1&wt=javabin&router.name=compositeId&replicationFactor=2&name=solrj_collection&action=CREATE&version=2
[junit4] 2> 112553 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112555 T314 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 112556 T314 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 112556 T314 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 112557 T314 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 112562 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112563 T313 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112564 T313 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
[junit4] 2> 112564 T313 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
[junit4] 2> 112568 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112570 T358 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112570 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112570 T312 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112570 T372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112570 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112570 T330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112662 T314 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
[junit4] 2> 112662 T314 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:52002_
[junit4] 2> 112666 T314 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:37596_
[junit4] 2> 112666 T314 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:40862_
[junit4] 2> 112666 T314 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:40421_
[junit4] 2> 112674 T308 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down
[junit4] 2> 112674 T323 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down
[junit4] 2> 112674 T367 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down
[junit4] 2> 112674 T353 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down
[junit4] 2> 112676 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112676 T323 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 112677 T367 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 112677 T353 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 112677 T308 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 112678 T313 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 112679 T313 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37596",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37596_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 112684 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112686 T313 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40421",
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40421_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 112689 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112690 T313 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52002",
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52002_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 112693 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112694 T313 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40862",
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40862_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 112697 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 112801 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112801 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112801 T372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112801 T358 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112801 T312 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 112801 T330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 113676 T323 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 113677 T367 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 113677 T353 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 113678 T323 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solr.xml
[junit4] 2> 113678 T353 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solr.xml
[junit4] 2> 113679 T353 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1
[junit4] 2> 113679 T353 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 113678 T323 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2
[junit4] 2> 113679 T323 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 113678 T308 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 113678 T367 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solr.xml
[junit4] 2> 113680 T308 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solr.xml
[junit4] 2> 113680 T353 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 113681 T353 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 113681 T308 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2
[junit4] 2> 113680 T323 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 113682 T323 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 113680 T367 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1
[junit4] 2> 113682 T367 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 113681 T308 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 113682 T353 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/'
[junit4] 2> 113687 T367 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 113687 T323 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/'
[junit4] 2> 113687 T308 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 113688 T308 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 113687 T367 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 113691 T367 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/'
[junit4] 2> 113691 T308 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/'
[junit4] 2> 113739 T353 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 113742 T323 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 113754 T367 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 113759 T308 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 113790 T308 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 113791 T353 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 113792 T308 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 113793 T353 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 113799 T323 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 113801 T367 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 113801 T323 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 113802 T367 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 113807 T308 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 113810 T323 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 113811 T353 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 113815 T367 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 114237 T308 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114239 T308 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114241 T308 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 114257 T308 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 114259 T308 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 114279 T353 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114280 T353 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114279 T323 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114281 T323 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114281 T353 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 114282 T323 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 114290 T353 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 114293 T353 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 114294 T367 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114294 T367 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 114296 T367 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 114296 T323 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 114297 T308 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114299 T323 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 114302 T308 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114307 T367 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 114307 T308 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114310 T367 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 114312 T308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114312 T308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114313 T308 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114318 T308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114319 T308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114319 T308 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 114320 T308 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/data/
[junit4] 2> 114320 T308 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2da0474e
[junit4] 2> 114321 T308 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/data
[junit4] 2> 114322 T308 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/data/index/
[junit4] 2> 114322 T308 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 114322 T308 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389069650018/solrj_collection_shard2_replica2/data/index
[junit4] 2> 114323 T308 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8297709384753379]
[junit4] 2> 114324 T308 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@f21ba12 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ae7d376),segFN=segments_1,generation=1}
[junit4] 2> 114324 T308 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114330 T353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114339 T308 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 114339 T308 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 114340 T353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114340 T308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 114340 T308 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 114341 T308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 114341 T308 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 114342 T308 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114342 T308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114343 T308 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 114343 T367 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114344 T308 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 114351 T308 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 114352 T308 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 114353 T308 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 114353 T353 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114355 T367 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114356 T323 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114354 T308 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 114357 T308 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 114358 T353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114358 T308 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 114358 T353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114359 T353 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114361 T353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114361 T353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114361 T308 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 114361 T323 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 114362 T367 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114362 T353 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 114363 T353 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/data/
[junit4] 2> 114364 T353 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2da0474e
[junit4] 2> 114366 T353 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/data
[junit4] 2> 114366 T367 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114367 T367 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114366 T308 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 114368 T308 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 114367 T367 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114367 T353 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 114369 T353 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 114369 T323 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114369 T353 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389069652521/solrj_collection_shard2_replica1/data/index
[junit4] 2> 114369 T367 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114370 T367 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114370 T367 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 114370 T353 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8297709384753379]
[junit4] 2> 114371 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114371 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114370 T367 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/data/
[junit4] 2> 114371 T323 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 114371 T308 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=66.9921875, floorSegmentMB=1.248046875, forceMergeDeletesPctAllowed=24.861097732982987, segmentsPerTier=39.0, maxCFSSegmentSizeMB=1.3095703125, noCFSRatio=1.0
[junit4] 2> 114372 T353 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@44252d6f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be0c257),segFN=segments_1,generation=1}
[junit4] 2> 114372 T353 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114372 T367 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2da0474e
[junit4] 2> 114373 T308 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@f21ba12 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ae7d376),segFN=segments_1,generation=1}
[junit4] 2> 114373 T308 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114374 T308 oass.SolrIndexSearcher.<init> Opening Searcher@760e2989 main
[junit4] 2> 114374 T367 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/data
[junit4] 2> 114374 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 114375 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 114375 T367 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/data/index/
[junit4] 2> 114375 T323 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 114375 T367 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 114376 T323 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/data/
[junit4] 2> 114376 T367 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/solrj_collection_shard1_replica1/data/index
[junit4] 2> 114376 T323 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2da0474e
[junit4] 2> 114377 T367 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8297709384753379]
[junit4] 2> 114377 T323 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/data
[junit4] 2> 114377 T367 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4291d43d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@321dc945),segFN=segments_1,generation=1}
[junit4] 2> 114377 T323 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 114378 T367 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114378 T323 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 114379 T323 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389069650216/solrj_collection_shard1_replica2/data/index
[junit4] 2> 114380 T378 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica2] Registered new searcher Searcher@760e2989 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 114380 T323 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8297709384753379]
[junit4] 2> 114381 T353 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 114382 T353 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 114382 T323 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3c2686b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a270750),segFN=segments_1,generation=1}
[junit4] 2> 114382 T323 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114382 T353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 114383 T353 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 114383 T353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 114383 T353 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 114384 T353 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114384 T353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114384 T353 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 114385 T308 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica2
[junit4] 2> 114385 T353 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 114385 T308 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica2 address:http://127.0.0.1:40421 collection:solrj_collection shard:shard2
[junit4] 2> 114386 T353 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 114386 T353 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 114387 T353 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 114387 T353 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 114388 T353 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 114388 T308 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard2/election
[junit4] 2> 114388 T353 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 114389 T367 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 114389 T367 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 114390 T367 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 114390 T353 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 114390 T367 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 114391 T367 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 114391 T323 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 114391 T367 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 114392 T323 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 114392 T367 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114393 T323 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 114393 T353 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 114393 T367 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114393 T353 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 114393 T323 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 114394 T367 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 114394 T323 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 114395 T353 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=66.9921875, floorSegmentMB=1.248046875, forceMergeDeletesPctAllowed=24.861097732982987, segmentsPerTier=39.0, maxCFSSegmentSizeMB=1.3095703125, noCFSRatio=1.0
[junit4] 2> 114395 T367 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 114395 T323 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 114396 T353 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@44252d6f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be0c257),segFN=segments_1,generation=1}
[junit4] 2> 114396 T367 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 114397 T367 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 114397 T323 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114397 T353 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 114398 T323 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 114397 T367 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 114398 T323 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 114398 T353 oass.SolrIndexSearcher.<init> Opening Searcher@71d9d448 main
[junit4] 2> 114399 T367 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 114400 T323 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 114401 T367 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 114401 T308 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 114402 T367 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 114402 T323 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 114403 T323 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 114403 T323 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 114403 T367 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 114404 T323 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 114405 T323 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 114405 T380 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica1] Registered new searcher Searcher@71d9d448 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 114406 T367 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 114406 T323 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 114406 T312 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 114406 T367 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 114407 T308 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=-1
[junit4] 2> 114407 T308 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Was waiting for r
[...truncated too long message...]
xWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 231789 T372 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 231789 T372 oasc.SolrCore.closeSearcher [nodes_used_collection_shard1_replica1] Closing main searcher on request.
[junit4] 2> 231789 T372 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 231790 T372 oasc.CachingDirectoryFactory.closeCacheValue looking to close ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data [CachedDir<<refCount=0;path=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data;done=false>>]
[junit4] 2> 231790 T372 oasc.CachingDirectoryFactory.close Closing directory: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data
[junit4] 2> 231790 T372 oasc.CachingDirectoryFactory.closeCacheValue looking to close ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data/index [CachedDir<<refCount=0;path=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data/index;done=false>>]
[junit4] 2> 231790 T372 oasc.CachingDirectoryFactory.close Closing directory: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389069653673/nodes_used_collection_shard1_replica1/data/index
[junit4] 2> 231790 T372 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4] 2> 231791 T372 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 231791 T372 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/nodes_used_collection/leader_elect/shard1/election/91034068567392270-core_node2-n_0000000001
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
[junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:276)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 231792 T372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 231792 T372 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 231792 T372 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 231792 T372 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_SY, timezone=Africa/Khartoum
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=2,free=119256128,total=330424320
[junit4] 2> NOTE: All tests run in this JVM: [TestStressReorder, SpellingQueryConverterTest, TestFieldResource, ConvertedLegacyTest, HdfsChaosMonkeySafeLeaderTest, TestSolrJ, SolrCmdDistributorTest, TestImplicitCoreProperties, TestDynamicFieldCollectionResource, TestSimpleQParserPlugin, TestFastWriter, BadIndexSchemaTest, BasicZkTest, TestCollationField, CollectionsAPIDistributedZkTest]
[junit4] Completed on J1 in 124.53s, 1 test, 1 failure <<< FAILURES!
[...truncated 928 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1307: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:940: There were test failures: 348 suites, 1522 tests, 1 failure, 36 ignored (7 assumptions)
Total time: 42 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b02 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_45) - Build # 8895
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8895/
Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
Stack Trace:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at __randomizedtesting.SeedInfo.seed([EEE461B526A4C275:EAECEE4634012D54]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:428)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:425)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:277)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9593 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 401650 T1232 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1389082392482
[junit4] 2> 401651 T1232 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 401660 T1232 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 401660 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 401661 T1233 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 401761 T1232 oasc.ZkTestServer.run start zk server on port:51637
[junit4] 2> 401766 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 401856 T1239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40c0fca2 name:ZooKeeperConnection Watcher:127.0.0.1:51637/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 401856 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 401857 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 401859 T1241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bf67368 name:ZooKeeperConnection Watcher:127.0.0.1:51637 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 401860 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 401871 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 401873 T1243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51e08af7 name:ZooKeeperConnection Watcher:127.0.0.1:51637 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 401873 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 401874 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 401885 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 401895 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 401904 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 401911 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 401917 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 401918 T1245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e922fa name:ZooKeeperConnection Watcher:127.0.0.1:51637/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 401919 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 401922 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 401929 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 401941 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 401948 T1232 oasc.Overseer.start Overseer (id=91034903675273219-127.0.0.1:51637_solr-n_0000000000) starting
[junit4] 2> 401966 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 401980 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 401991 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 402005 T1246 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 402010 T1247 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 402016 T1245 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402018 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 402021 T1246 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 402022 T1246 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 402022 T1246 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 402032 T1245 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402033 T1239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 402075 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 402078 T1245 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402082 T1246 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 402083 T1246 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 402084 T1246 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 402084 T1246 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 402089 T1245 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402197 T1239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 402237 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51637 51637
[junit4] 2> 402337 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 402343 T1232 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 402343 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 402344 T1248 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 402444 T1232 oasc.ZkTestServer.run start zk server on port:34843
[junit4] 2> 402456 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 402461 T1254 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7abd275b name:ZooKeeperConnection Watcher:127.0.0.1:34843 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 402461 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 402504 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 402506 T1256 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6af81f79 name:ZooKeeperConnection Watcher:127.0.0.1:34843 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 402506 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 402507 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 402523 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 402531 T1258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53ee9231 name:ZooKeeperConnection Watcher:127.0.0.1:34843/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 402531 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 402532 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 402540 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 402546 T1260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dda0a1f name:ZooKeeperConnection Watcher:127.0.0.1:34843/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 402546 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 402551 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 402559 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 402576 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 402580 T1232 oasc.Overseer.start Overseer (id=91034903719444483-127.0.0.1:34843_solr-n_0000000000) starting
[junit4] 2> 402594 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 402601 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 402608 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 402613 T1262 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 402616 T1261 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 402614 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 402637 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 402643 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 402650 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 402657 T1264 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1731d9aa name:ZooKeeperConnection Watcher:127.0.0.1:34843/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 402658 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 402667 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 402672 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 402690 T1258 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 402691 T1264 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 402694 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402697 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 402698 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 402698 T1261 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 402698 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 402702 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 402704 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 402704 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 403197 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 403219 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 403232 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403238 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403239 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 403246 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 403246 T1261 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 403247 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 403250 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403355 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 403355 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 403739 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 403760 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 403776 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403777 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403777 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403781 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403781 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 403799 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 403799 T1261 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 403799 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 403803 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 403907 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 403908 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 404282 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 404301 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 404318 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404319 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404319 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404322 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 404326 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404337 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 404337 T1261 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 404338 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 404343 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404448 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 404448 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 404826 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 404843 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404843 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404843 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404846 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 404847 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 404848 T1261 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 404848 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 404852 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 404957 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 404958 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 405343 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 405355 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 405356 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 405356 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 405358 T1261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 405360 T1261 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 405360 T1261 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 405360 T1261 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 405370 T1260 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 405476 T1264 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 405476 T1258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 405856 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 405873 T1264 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 405877 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34843 34843
[junit4] 2> 406018 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 406028 T1232 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
[junit4] 2> 406028 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 406031 T1265 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 406131 T1232 oasc.ZkTestServer.run start zk server on port:48339
[junit4] 2> 406138 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 406147 T1271 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@789bb6af name:ZooKeeperConnection Watcher:127.0.0.1:48339/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 406147 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 406153 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 406154 T1273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71374662 name:ZooKeeperConnection Watcher:127.0.0.1:48339 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 406154 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 406258 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 406261 T1275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e4adccc name:ZooKeeperConnection Watcher:127.0.0.1:48339 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 406263 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 406264 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 406269 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 406280 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 406287 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 406290 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 406304 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 406314 T1277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e9248dd name:ZooKeeperConnection Watcher:127.0.0.1:48339/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 406315 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 406317 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 406324 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 406339 T1271 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 406339 T1277 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 406340 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 406342 T1279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@286445eb name:ZooKeeperConnection Watcher:127.0.0.1:48339/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 406342 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 406344 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 406358 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 406368 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 406374 T1232 oasc.Overseer.start Overseer (id=91034903961796612-127.0.0.1:48339_solr-n_0000000000) starting
[junit4] 2> 406379 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 406388 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 406405 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 406417 T1281 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 406421 T1280 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 406426 T1280 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 406427 T1280 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 406428 T1280 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 406428 T1280 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 406433 T1279 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 406435 T1271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 406435 T1277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 406922 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 406938 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 406956 T1279 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 406958 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 406960 T1280 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 406973 T1271 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 406977 T1277 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 406991 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48339 48339
[junit4] 2> 407059 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 407066 T1232 oas.SolrTestCaseJ4.setUp ###Starting testDoubleAssignment
[junit4] 2> 407066 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 407067 T1282 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 407167 T1232 oasc.ZkTestServer.run start zk server on port:38530
[junit4] 2> 407170 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407179 T1288 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6842133f name:ZooKeeperConnection Watcher:127.0.0.1:38530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407179 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407181 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407183 T1290 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14027b82 name:ZooKeeperConnection Watcher:127.0.0.1:38530 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407184 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407271 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407279 T1292 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5343df1b name:ZooKeeperConnection Watcher:127.0.0.1:38530 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407279 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407279 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 407297 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 407297 T1283 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1436bc34cd30002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 407305 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 407309 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 407322 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 407328 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407329 T1294 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@127bb2ee name:ZooKeeperConnection Watcher:127.0.0.1:38530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407330 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407332 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 407339 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 407344 T1288 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 407346 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407349 T1294 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 407350 T1296 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@502023c2 name:ZooKeeperConnection Watcher:127.0.0.1:38530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407350 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407353 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 407363 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 407371 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 407378 T1264 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 407379 T1264 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 407380 T1264 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 407382 T1232 oasc.Overseer.start Overseer (id=91034904029233156-127.0.0.1:38530_solr-n_0000000000) starting
[junit4] 2> 407387 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 407393 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 407401 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 407408 T1298 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 407411 T1297 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 407415 T1297 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 407416 T1297 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 407416 T1297 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 407417 T1297 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 407422 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 407424 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 407424 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 407913 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 407931 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 407945 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 407946 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 407951 T1297 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 407953 T1288 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 407958 T1294 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 407961 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 407965 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 407965 T1300 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e91cae name:ZooKeeperConnection Watcher:127.0.0.1:38530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 407965 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 407967 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 407972 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 407979 T1288 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 407979 T1300 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 407983 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 407984 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 407986 T1297 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 407987 T1297 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 407992 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 407999 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 408008 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 408020 T1296 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 408127 T1300 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 408128 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 408137 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 408151 T1288 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 408151 T1300 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 408159 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:38530 38530
[junit4] 2> 408304 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 408311 T1232 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
[junit4] 2> 408312 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 408312 T1301 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 408412 T1232 oasc.ZkTestServer.run start zk server on port:46940
[junit4] 2> 408417 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 408420 T1307 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f074546 name:ZooKeeperConnection Watcher:127.0.0.1:46940/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 408420 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 408421 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 408437 T1309 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fbcd2c name:ZooKeeperConnection Watcher:127.0.0.1:46940 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 408438 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 408501 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 408505 T1311 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52955821 name:ZooKeeperConnection Watcher:127.0.0.1:46940 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 408505 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 408506 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 408516 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 408520 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 408531 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 408535 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 408546 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 408549 T1313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@365f4402 name:ZooKeeperConnection Watcher:127.0.0.1:46940/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 408549 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 408552 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 408558 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 408562 T1307 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 408565 T1313 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 408566 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 408567 T1315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a00f7ac name:ZooKeeperConnection Watcher:127.0.0.1:46940/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 408568 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 408569 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 408584 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 408593 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 408599 T1232 oasc.Overseer.start Overseer (id=91034904110825476-127.0.0.1:46940_solr-n_0000000000) starting
[junit4] 2> 408604 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 408614 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 408631 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 408643 T1317 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 408656 T1316 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 409647 T1315 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 409650 T1316 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 409651 T1316 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 409651 T1316 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 409652 T1316 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 409656 T1315 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 409658 T1313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 409658 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 410148 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 410165 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 410176 T1315 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 410178 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 410181 T1316 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 410189 T1315 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 410190 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 410195 T1316 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 410196 T1316 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 410207 T1315 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 410312 T1313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 410315 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411376 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 411394 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 411395 T1319 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b1c2fd2 name:ZooKeeperConnection Watcher:127.0.0.1:46940/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411395 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 411406 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 411415 T1232 oasc.Overseer.start Overseer (id=91034904110825477-127.0.0.1:46940_solr-n_0000000001) starting
[junit4] 2> 411425 T1321 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 411436 T1320 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 411442 T1320 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 411444 T1320 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 411444 T1320 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 411449 T1319 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 411460 T1313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411461 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411517 T1319 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 411517 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 411524 T1320 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 411542 T1320 oasc.SolrException.log ERROR Problem cleaning up collection in zk:collection1:org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:654)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:649)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:649)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:649)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:875)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:226)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 411549 T1313 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 411550 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 411553 T1319 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 411569 T1307 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 411575 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46940 46940
[junit4] 2> 411654 T1320 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:89)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:411)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:195)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 411748 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=EEE461B526A4C275 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el_CY -Dtests.timezone=Antarctica/South_Pole -Dtests.file.encoding=UTF-8
[junit4] ERROR 3.48s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] > at __randomizedtesting.SeedInfo.seed([EEE461B526A4C275:EAECEE4634012D54]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:428)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:425)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
[junit4] > at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:277)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
[junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4] 2> 411796 T1232 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 411796 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 411797 T1322 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 411903 T1232 oasc.ZkTestServer.run start zk server on port:38540
[junit4] 2> 411904 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 411916 T1328 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f6eec9b name:ZooKeeperConnection Watcher:127.0.0.1:38540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411916 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 411919 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 411937 T1330 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c95b412 name:ZooKeeperConnection Watcher:127.0.0.1:38540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411938 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 411990 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412000 T1332 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a7a5452 name:ZooKeeperConnection Watcher:127.0.0.1:38540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412000 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412000 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 412010 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 412014 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 412018 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 412022 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 412027 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 412049 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412050 T1334 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a2edb9 name:ZooKeeperConnection Watcher:127.0.0.1:38540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412050 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412051 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 412055 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 412072 T1232 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 412077 T1232 oasc.Overseer.start Overseer (id=91034904339546115-127.0.0.1:38540_solr-n_0000000000) starting
[junit4] 2> 412081 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 412089 T1232 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 412095 T1336 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 412111 T1335 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 412112 T1335 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 412113 T1335 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 412115 T1328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 412135 T1335 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 412136 T1328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 412141 T1335 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 412144 T1335 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 412151 T1335 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 412162 T1334 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412164 T1328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 412212 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:38540 38540
[junit4] 2> 412426 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 412432 T1232 oas.SolrTestCaseJ4.setUp ###Starting testShardLeaderChange
[junit4] 2> 412433 T1232 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 412433 T1337 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 412533 T1232 oasc.ZkTestServer.run start zk server on port:60184
[junit4] 2> 412534 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412543 T1343 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f0480e1 name:ZooKeeperConnection Watcher:127.0.0.1:60184/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412544 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412545 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412546 T1345 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53b2f6d4 name:ZooKeeperConnection Watcher:127.0.0.1:60184 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412547 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412585 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412586 T1347 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@717a09a6 name:ZooKeeperConnection Watcher:127.0.0.1:60184 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412587 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412587 T1232 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 412601 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 412606 T1348 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412607 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412610 T1350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bf90a83 name:ZooKeeperConnection Watcher:127.0.0.1:60184/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412610 T1348 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412610 T1352 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b212ccd name:ZooKeeperConnection Watcher:127.0.0.1:60184/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412611 T1348 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 412616 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412618 T1232 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 412623 T1348 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 412627 T1232 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 412630 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 412638 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 412643 T1348 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 412653 T1348 oasc.Overseer.start Overseer (id=91034904380899331-127.0.0.1:60184_solr-n_0000000000) starting
[junit4] 2> 412653 T1352 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 412655 T1232 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 412662 T1348 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 412678 T1348 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 412684 T1348 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 412692 T1354 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 412692 T1348 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 412695 T1353 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer work queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:89)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.element(DistributedQueue.java:131)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:325)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:105)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 412695 T1348 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412708 T1356 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59a9d2d name:ZooKeeperConnection Watcher:127.0.0.1:60184/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412708 T1348 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412712 T1348 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 412725 T1348 oasc.Overseer.start Overseer (id=91034904380899333-127.0.0.1:60184_solr-n_0000000001) starting
[junit4] 2> 412745 T1358 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 412748 T1357 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 412755 T1357 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 412756 T1357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 412757 T1357 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 412757 T1357 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 412763 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412765 T1352 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 413070 T1313 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:239)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:236)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:236)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:284)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:271)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:121)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 413071 T1313 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 413072 T1313 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 413162 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 413182 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 413191 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413195 T1357 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 413198 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413198 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 413217 T1357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 413217 T1357 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 413231 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413233 T1232 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 413242 T1360 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@174c3ff1 name:ZooKeeperConnection Watcher:127.0.0.1:60184/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 413242 T1232 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 413259 T1232 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 413263 T1232 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 413267 T1352 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 413270 T1360 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 413276 T1232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 413276 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413283 T1357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 413283 T1357 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 413288 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413292 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 413392 T1360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 413392 T1352 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 413495 T1232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 413504 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413504 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413505 T1356 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413508 T1357 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 413510 T1357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "s
[...truncated too long message...]
ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 435305 T1643 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 435311 T1645 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 435315 T1647 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 435315 T1645 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 435317 T1651 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 435317 T1653 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 435321 T1651 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 435321 T1647 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 435322 T1649 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 435322 T1653 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 435326 T1651 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 435327 T1649 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 435327 T1653 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 435330 T1653 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 435330 T1651 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 435334 T1653 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 435335 T1232 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34523 34523
[junit4] 2> 435454 T1232 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignmentBigger
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 435456 T1232 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 435456 T1232 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 435705 T1561 oasc.Overseer.createOverseerNode ERROR Could not create Overseer node org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:305)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:305)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1037)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1019)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:121)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 435706 T1561 oasc.LeaderElector$1.process WARN java.lang.RuntimeException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1046)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1019)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:121)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:305)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:305)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1037)
[junit4] 2> ... 8 more
[junit4] 2>
[junit4] 2> 435706 T1561 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 436145 T1585 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 436670 T1623 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 438456 T1232 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=el_CY, timezone=Antarctica/South_Pole
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=8,threads=1,free=57732064,total=103424000
[junit4] 2> NOTE: All tests run in this JVM: [TestPHPSerializedResponseWriter, TestJoin, TestStressVersions, JsonLoaderTest, TestValueSourceCache, TriLevelCompositeIdRoutingTest, QueryParsingTest, StatelessScriptUpdateProcessorFactoryTest, SoftAutoCommitTest, TestRemoteStreaming, BadCopyFieldTest, ClusterStateUpdateTest, DocumentBuilderTest, TestLRUCache, SyncSliceTest, SolrTestCaseJ4Test, TestLMJelinekMercerSimilarityFactory, TestDynamicFieldResource, TestJmxMonitoredMap, DOMUtilTest, DirectUpdateHandlerOptimizeTest, FileBasedSpellCheckerTest, ResourceLoaderTest, TestSearchPerf, TestDocumentBuilder, CopyFieldTest, TestFieldTypeCollectionResource, QueryElevationComponentTest, DistributedDebugComponentTest, TestSerializedLuceneMatchVersion, TestSolrXMLSerializer, InfoHandlerTest, TestSystemIdResolver, TestAddFieldRealTimeGet, RegexBoostProcessorTest, DocValuesTest, DateMathParserTest, DeleteShardTest, OverseerCollectionProcessorTest, SystemInfoHandlerTest, TestSolrQueryParser, TestCollationFieldDocValues, PeerSyncTest, SearchHandlerTest, OverseerTest]
[junit4] Completed on J0 in 36.87s, 8 tests, 1 error <<< FAILURES!
[...truncated 724 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1307: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:940: There were test failures: 348 suites, 1522 tests, 1 error, 36 ignored (7 assumptions)
Total time: 49 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b02) - Build
# 8894 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8894/
Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
Stack Trace:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at __randomizedtesting.SeedInfo.seed([B759668F57E91878:B351E97C454CF759]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:428)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:425)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:277)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 9979 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 17406 T119 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1389075934881
[junit4] 2> 17407 T119 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 17412 T119 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 17415 T119 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 17417 T120 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 17516 T119 oasc.ZkTestServer.run start zk server on port:35291
[junit4] 2> 17582 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 17686 T126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64ecb973 name:ZooKeeperConnection Watcher:127.0.0.1:35291/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 17687 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 17689 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 17692 T128 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2477058e name:ZooKeeperConnection Watcher:127.0.0.1:35291 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 17692 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 17720 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 17721 T121 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1436b60aeec0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 17725 T130 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e5bcd5a name:ZooKeeperConnection Watcher:127.0.0.1:35291 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 17725 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 17727 T119 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 17738 T119 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 17747 T119 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 17751 T119 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 17755 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 17780 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 17782 T132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@341c8be2 name:ZooKeeperConnection Watcher:127.0.0.1:35291/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 17783 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 17787 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 17793 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 17814 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 17819 T119 oasc.Overseer.start Overseer (id=91034480473276419-127.0.0.1:35291_solr-n_0000000000) starting
[junit4] 2> 17832 T119 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 17838 T119 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 17857 T119 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 17871 T134 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 17877 T119 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 17879 T133 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 17885 T133 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 17887 T133 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 17889 T133 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 17902 T132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 17910 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 17931 T119 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 17934 T132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 17936 T133 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 17937 T133 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 17938 T133 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 17938 T133 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 17942 T132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 18045 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 18089 T119 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:35291 35291
[junit4] 2> 18165 T119 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 18174 T119 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 18175 T119 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 18175 T135 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 18275 T119 oasc.ZkTestServer.run start zk server on port:46955
[junit4] 2> 18276 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18279 T141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@183d330c name:ZooKeeperConnection Watcher:127.0.0.1:46955 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18279 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18298 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18300 T143 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7cf92417 name:ZooKeeperConnection Watcher:127.0.0.1:46955 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18300 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18301 T119 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 18305 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18306 T145 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66fd8b97 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18307 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18307 T119 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 18310 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18312 T147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ee5c773 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18312 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18314 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 18317 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 18323 T119 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 18326 T119 oasc.Overseer.start Overseer (id=91034480519086083-127.0.0.1:46955_solr-n_0000000000) starting
[junit4] 2> 18329 T119 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 18334 T119 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 18339 T119 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 18343 T149 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 18344 T119 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 18345 T148 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 18354 T119 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 18357 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18362 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18369 T151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@462cd2d6 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18369 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18371 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18375 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 18378 T145 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 18378 T151 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 18379 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18380 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bacd694 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18381 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18382 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18385 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 18388 T153 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 18388 T145 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 18389 T151 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 18389 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18390 T155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@444f0429 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18391 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18392 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18396 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 18400 T151 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 18400 T153 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 18401 T145 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 18401 T155 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 18402 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18403 T157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1517f0c name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18403 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18405 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18408 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 18412 T151 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 18412 T145 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 18412 T157 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 18412 T153 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 18413 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18415 T155 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 18416 T159 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@168ef9df name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18417 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18418 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18422 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 18427 T155 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18427 T157 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18427 T159 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18427 T145 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18427 T151 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18428 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18427 T153 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 18430 T161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44a0abc2 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18430 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18431 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18435 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 18438 T151 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T157 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T155 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T145 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T159 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T161 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18439 T153 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 18440 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18442 T163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@308cd450 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18442 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18444 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18447 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 18451 T157 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18451 T145 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18451 T159 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18451 T155 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18452 T153 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18452 T161 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18452 T163 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18452 T151 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 18454 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18455 T165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e5f3f90 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18456 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18457 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18460 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 18464 T153 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18464 T161 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18465 T157 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18465 T165 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18466 T155 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18466 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18464 T151 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18466 T145 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18466 T163 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18466 T159 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 18468 T167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@374eb126 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18468 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18470 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18473 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 18478 T151 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18478 T157 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18478 T155 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18478 T167 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18478 T153 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18479 T145 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18479 T165 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18479 T161 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18480 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18480 T159 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18480 T163 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 18482 T169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b11f959 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18482 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18484 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18488 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 18492 T151 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18492 T159 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18493 T145 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18493 T155 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18493 T161 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18492 T157 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18493 T163 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18493 T165 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18494 T167 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18494 T153 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18494 T169 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 18495 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18497 T171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e62c4bb name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18497 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18499 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18502 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 18506 T157 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18506 T145 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T161 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T155 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T163 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T165 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T153 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T151 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18508 T171 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18508 T169 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T159 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18507 T167 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 18508 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18510 T173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1508dd9 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18510 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18512 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18516 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 18520 T157 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18521 T163 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18521 T165 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18521 T153 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18522 T151 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18522 T167 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18522 T159 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18522 T161 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18522 T155 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18523 T145 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18523 T169 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18523 T173 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18524 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18526 T175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7028bca4 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18526 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18528 T171 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 18528 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18532 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 18536 T171 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18536 T173 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18537 T145 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18537 T157 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18538 T161 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18538 T165 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18538 T163 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18538 T167 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18539 T151 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18539 T169 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18538 T159 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18539 T175 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18539 T155 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18540 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18539 T153 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 18541 T177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@359fe30c name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18542 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18543 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18546 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 18551 T151 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18551 T167 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18552 T169 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18552 T175 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18551 T165 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18552 T155 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18552 T153 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18553 T171 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18553 T145 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18553 T163 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18552 T161 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18554 T177 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18554 T159 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18554 T173 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18553 T157 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 18554 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18556 T179 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7aca0a76 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18556 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18558 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18562 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 18568 T151 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18568 T159 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18568 T169 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18569 T165 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18568 T145 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18569 T171 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18569 T161 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18569 T175 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18570 T163 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18570 T157 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18570 T173 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18570 T179 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18571 T167 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18571 T155 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18571 T153 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18571 T177 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 18572 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18574 T181 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60efa746 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18574 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18576 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18580 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 18584 T169 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18584 T175 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18585 T153 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18585 T157 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18585 T155 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18585 T161 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18586 T165 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18586 T167 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18586 T151 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T163 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T179 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T159 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T145 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18588 T177 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T181 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T173 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18587 T171 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 18588 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18590 T183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@312679c9 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18590 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18592 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18595 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 18600 T163 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18600 T155 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18600 T169 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18600 T171 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18601 T153 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18601 T183 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18603 T167 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18603 T179 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18601 T175 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18604 T157 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18603 T181 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18603 T165 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18602 T173 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18602 T161 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18602 T159 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18602 T145 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18601 T177 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18601 T151 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 18612 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18614 T185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47ccf71d name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18614 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18616 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18620 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 18626 T159 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T153 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T145 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T175 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T179 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T161 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18627 T165 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T183 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T173 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T169 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T163 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T171 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T151 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T157 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18630 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18629 T185 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18629 T181 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T167 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T155 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18628 T177 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 18632 T187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eb25fe5 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18632 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18634 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18638 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 18642 T165 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18643 T153 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T151 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T185 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T163 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18645 T187 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18645 T169 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18645 T171 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T159 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T145 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T167 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18646 T161 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18645 T181 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18645 T177 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18644 T173 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18648 T175 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18648 T179 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18647 T183 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18647 T157 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18647 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18647 T155 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 18650 T189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d169bef name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18650 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18652 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18655 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 18660 T165 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18660 T173 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18660 T187 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18661 T155 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18660 T167 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18661 T183 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18662 T185 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18662 T161 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18662 T189 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18660 T177 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18663 T175 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18663 T169 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18664 T145 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18664 T181 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18662 T157 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18662 T159 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18664 T151 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18665 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18664 T153 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18664 T163 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18663 T179 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18663 T171 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 18667 T191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@473d9d9a name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18667 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18669 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18672 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 18676 T157 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18677 T155 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18677 T171 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18678 T145 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18678 T173 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18678 T167 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18678 T181 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18679 T185 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18679 T183 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18679 T153 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18680 T175 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18680 T177 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18680 T161 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18681 T163 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18680 T169 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18681 T159 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18681 T187 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18682 T189 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18682 T191 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18682 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18682 T151 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18682 T165 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18682 T179 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 18684 T193 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36077c7 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18684 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18686 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18690 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 18696 T157 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18696 T189 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18696 T187 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18697 T155 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18697 T159 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18698 T185 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18697 T163 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18697 T177 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18698 T181 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18698 T173 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18698 T167 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18699 T183 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18698 T171 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18697 T179 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T161 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T145 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T191 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T153 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18699 T151 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18699 T165 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T169 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T175 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18700 T193 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 18701 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18703 T195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2aa1e1d8 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18704 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18705 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18708 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 18715 T165 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18715 T153 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18716 T175 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18716 T151 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18716 T169 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18716 T179 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18717 T161 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18717 T185 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18717 T193 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18717 T191 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18718 T159 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18718 T167 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18718 T195 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18718 T155 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18719 T145 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18719 T187 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18719 T181 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18719 T177 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18720 T189 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18719 T157 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18720 T171 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18720 T163 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18720 T183 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18720 T173 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 18723 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18725 T197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f2e3500 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18725 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18727 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18730 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 18737 T173 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18737 T171 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18737 T181 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18737 T183 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18738 T153 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18737 T195 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18738 T187 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18738 T177 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18738 T165 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18739 T191 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18739 T161 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18741 T169 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18740 T151 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18741 T157 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18740 T175 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18740 T155 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18740 T185 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18740 T163 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T197 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T159 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T179 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T193 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T145 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18742 T189 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18741 T167 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 18743 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18745 T199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9e37ff4 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18745 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18746 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18749 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 18754 T151 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18755 T169 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18755 T175 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18755 T193 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T167 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T159 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18757 T157 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T145 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18757 T161 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T179 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18757 T197 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18757 T185 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18757 T163 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T155 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18756 T189 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18759 T195 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18759 T181 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18760 T177 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18759 T171 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18758 T187 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18758 T191 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18758 T165 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18758 T183 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18758 T153 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18760 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18760 T199 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18759 T173 oascc.ZkStateReader$3.process Updating live nodes... (25)
[junit4] 2> 18762 T201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fdbf01e name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18763 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18764 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18767 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 18771 T195 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18772 T173 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18773 T181 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18773 T155 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18773 T153 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18773 T183 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T157 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T187 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T175 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T163 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T197 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T193 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T161 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18773 T189 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T145 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18776 T165 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T171 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T191 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T159 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18775 T177 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T169 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T167 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18774 T185 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18778 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18776 T199 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18776 T201 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18776 T179 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18776 T151 oascc.ZkStateReader$3.process Updating live nodes... (26)
[junit4] 2> 18779 T203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@166a3c8b name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18779 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18781 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18783 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 18789 T193 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18790 T169 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18789 T179 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18790 T159 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18790 T157 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18791 T189 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18791 T163 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18789 T201 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18792 T167 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18792 T155 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18791 T185 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18791 T197 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18793 T187 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18794 T173 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18794 T199 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18791 T151 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18790 T145 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18790 T175 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18795 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18795 T203 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18794 T171 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18794 T195 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18794 T183 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18793 T191 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18793 T181 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18793 T161 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18793 T177 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18792 T153 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18792 T165 oascc.ZkStateReader$3.process Updating live nodes... (27)
[junit4] 2> 18798 T205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f6d7cc7 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18799 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18800 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18804 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 18812 T157 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18812 T155 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18813 T189 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18814 T163 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18814 T167 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18814 T145 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18814 T159 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T199 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T185 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T169 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T201 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T193 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18816 T175 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18816 T151 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T181 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T171 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T203 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18815 T179 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18818 T187 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T205 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T177 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T173 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18817 T183 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18816 T197 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18819 T191 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18819 T153 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18819 T165 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18819 T195 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18818 T161 oascc.ZkStateReader$3.process Updating live nodes... (28)
[junit4] 2> 18822 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18822 T207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55b36784 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18823 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18824 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18827 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 18834 T167 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18834 T203 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18834 T185 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18834 T155 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18835 T175 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18835 T163 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T151 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T179 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T145 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18835 T169 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18835 T197 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T181 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T199 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T191 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18835 T157 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T173 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T165 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T171 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T177 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T183 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T159 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18840 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18837 T195 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18837 T187 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T201 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T193 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18836 T189 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18841 T209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9b9be7 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18841 T205 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18840 T207 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T161 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18838 T153 oascc.ZkStateReader$3.process Updating live nodes... (29)
[junit4] 2> 18842 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18844 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18857 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 18862 T165 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18863 T181 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18862 T191 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18865 T159 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18865 T155 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18864 T201 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18864 T177 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18863 T169 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18863 T151 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18863 T175 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T163 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T203 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T189 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T185 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T173 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T207 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18866 T193 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18866 T195 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18869 T179 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18866 T197 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18866 T157 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18865 T183 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18865 T153 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18869 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18869 T161 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T199 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T167 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T209 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T205 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T171 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18868 T187 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18867 T145 oascc.ZkStateReader$3.process Updating live nodes... (30)
[junit4] 2> 18871 T211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23339129 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18873 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18875 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18877 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 18886 T173 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18886 T181 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18887 T183 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18887 T187 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18887 T207 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18887 T177 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18887 T171 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T153 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T157 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T167 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T155 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T165 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T161 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18890 T145 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18890 T159 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T209 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T191 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18891 T211 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18890 T201 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18890 T179 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18890 T199 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T185 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T203 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T163 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18889 T189 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18888 T195 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18893 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18892 T175 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18892 T193 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18891 T197 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18891 T169 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18891 T151 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18891 T205 oascc.ZkStateReader$3.process Updating live nodes... (31)
[junit4] 2> 18895 T213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5aba985b name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18895 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18897 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18900 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 18906 T169 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18906 T181 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18906 T159 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18907 T157 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18907 T193 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18907 T189 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18907 T185 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T155 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T163 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18909 T171 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T175 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T167 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18909 T173 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18909 T207 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18909 T179 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T183 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T201 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T195 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18908 T145 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T209 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T187 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T205 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T211 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T177 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T197 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18910 T203 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18909 T151 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18912 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18912 T213 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T191 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T153 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T199 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T165 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18911 T161 oascc.ZkStateReader$3.process Updating live nodes... (32)
[junit4] 2> 18915 T215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@615a0685 name:ZooKeeperConnection Watcher:127.0.0.1:46955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 18915 T119 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 18916 T119 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 18918 T119 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 18928 T165 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18928 T209 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18928 T153 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18928 T191 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T207 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T213 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T161 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18928 T181 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18928 T195 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T211 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T151 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T175 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T145 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T197 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T169 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T201 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18932 T189 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T171 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T187 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T173 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T215 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18929 T183 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18933 T119 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 18932 T185 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18932 T167 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T179 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T155 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T157 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T163 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18931 T203 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T199 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T159 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T205 oascc.ZkStateReader$3.process Updating live nodes... (33)
[junit4] 2> 18930 T177 oascc.ZkStateReader$3.process Updating live nodes... (33)
[...truncated too long message...]
nChanged
[junit4] 2> 40130 T571 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 40131 T571 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 40131 T571 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 40131 T571 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 40133 T570 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40236 T568 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40236 T574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40629 T119 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 40635 T570 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40635 T570 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40635 T570 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40636 T571 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 40636 T571 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 40637 T571 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 40637 T571 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 40639 T570 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40742 T574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40742 T568 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 41135 T119 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 41145 T574 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 41150 T119 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:32886 32886
[junit4] 2> 41578 T119 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 41580 T119 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 41581 T119 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 42646 T574 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 42647 T574 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 42647 T574 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 44581 T119 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=es_MX, timezone=Asia/Amman
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=108643344,total=214433792
[junit4] 2> NOTE: All tests run in this JVM: [TestManagedSchema, TestLFUCache, LoggingHandlerTest, TestCollationFieldDocValues, TestNonNRTOpen, OverseerTest]
[junit4] Completed on J0 in 27.21s, 8 tests, 1 error <<< FAILURES!
[...truncated 1043 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1307: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:940: There were test failures: 348 suites, 1522 tests, 1 error, 35 ignored (6 assumptions)
Total time: 41 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure