You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/06/09 00:24:25 UTC
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b15) -
Build # 10509 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10509/
Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseG1GC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard2 is not consistent. Got 236 from https://127.0.0.1:50691/cdh/gy/collection1lastClient and got 215 from https://127.0.0.1:52172/cdh/gy/collection1
Stack Trace:
java.lang.AssertionError: shard2 is not consistent. Got 236 from https://127.0.0.1:50691/cdh/gy/collection1lastClient and got 215 from https://127.0.0.1:52172/cdh/gy/collection1
at __randomizedtesting.SeedInfo.seed([D7D1E61AF425929C:56376802837AF2A0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
createcollection the collection error [Watcher fired on path: null state: SyncConnected type None]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: createcollection the collection error [Watcher fired on path: null state: SyncConnected type None]
at __randomizedtesting.SeedInfo.seed([D7D1E61AF425929C:56376802837AF2A0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:566)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11070 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/init-core-data-001
[junit4] 2> 425904 T1794 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 425904 T1794 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /cdh/gy
[junit4] 2> 425907 T1794 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 425907 T1794 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 425908 T1795 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 426008 T1794 oasc.ZkTestServer.run start zk server on port:37566
[junit4] 2> 426010 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 426070 T1801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4660e1e5 name:ZooKeeperConnection Watcher:127.0.0.1:37566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 426071 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 426071 T1794 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 426077 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 426079 T1803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e7d7e8b name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 426080 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 426080 T1794 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 426085 T1794 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 426088 T1794 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 426091 T1794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 426095 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 426096 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 426101 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 426102 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 426106 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 426107 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 426111 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 426112 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 426115 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 426116 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 426118 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 426119 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 426123 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 426124 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 426127 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 426128 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 426131 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 426132 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 426135 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 426136 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 426140 T1794 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 426140 T1794 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 426146 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 426148 T1805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f84b3e6 name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 426148 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 426372 T1794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 426375 T1794 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 426378 T1794 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55629
[junit4] 2> 426379 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 426380 T1794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 426380 T1794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002
[junit4] 2> 426380 T1794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/'
[junit4] 2> 426391 T1794 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/solr.xml
[junit4] 2> 426399 T1794 oasc.CoreContainer.<init> New CoreContainer 2112156018
[junit4] 2> 426399 T1794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/]
[junit4] 2> 426400 T1794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 426400 T1794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 426401 T1794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 426401 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 426401 T1794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 426402 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 426402 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 426402 T1794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 426402 T1794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 426404 T1794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 426404 T1794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 426404 T1794 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 426405 T1794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37566/solr
[junit4] 2> 426405 T1794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 426405 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 426407 T1816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fbe795f name:ZooKeeperConnection Watcher:127.0.0.1:37566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 426408 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 426411 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 426413 T1818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dfd939a name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 426414 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 426416 T1794 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 426423 T1794 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 426430 T1794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 426434 T1794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 426438 T1794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 426442 T1794 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 426444 T1794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55629_cdh%2Fgy
[junit4] 2> 426446 T1794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55629_cdh%2Fgy
[junit4] 2> 426448 T1794 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 426450 T1794 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 426453 T1794 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:55629_cdh%2Fgy
[junit4] 2> 426453 T1794 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 426455 T1794 oasc.Overseer.start Overseer (id=91898822439403524-127.0.0.1:55629_cdh%2Fgy-n_0000000000) starting
[junit4] 2> 426457 T1794 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 426465 T1820 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 426466 T1794 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 426467 T1794 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 426468 T1794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 426470 T1819 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 426471 T1821 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 426471 T1821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 426472 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 426472 T1821 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 426473 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 426473 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55629/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55629_cdh%2Fgy",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 426473 T1819 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 426474 T1819 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 426475 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 426479 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 427472 T1821 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 427473 T1821 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 427474 T1821 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 427474 T1821 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 427476 T1821 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 427477 T1821 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/collection1/'
[junit4] 2> 427478 T1821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 427478 T1821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 427479 T1821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 427508 T1821 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 427518 T1821 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 427519 T1821 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 427524 T1821 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 427616 T1821 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 427617 T1821 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 427618 T1821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 427622 T1821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 427645 T1821 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 427645 T1821 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 427659 T1821 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/
[junit4] 2> 427659 T1821 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@463b09db
[junit4] 2> 427660 T1821 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data
[junit4] 2> 427661 T1821 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/index/
[junit4] 2> 427661 T1821 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 427662 T1821 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/index
[junit4] 2> 427663 T1821 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=7.9267578125, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=24.099611015611966, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4531112716772604
[junit4] 2> 427667 T1821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 427668 T1821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 427672 T1821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 427672 T1821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 427673 T1821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 427673 T1821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 427673 T1821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 427674 T1821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 427674 T1821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 427675 T1821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 427675 T1821 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 427675 T1821 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 427676 T1821 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 427676 T1821 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 427677 T1821 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 427677 T1821 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 427678 T1821 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 427678 T1821 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 427679 T1821 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 427681 T1821 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 427683 T1821 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 427683 T1821 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 427684 T1821 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=91.0322265625, floorSegmentMB=1.265625, forceMergeDeletesPctAllowed=17.181881993242474, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.1953125, noCFSRatio=1.0
[junit4] 2> 427685 T1821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 427686 T1821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 427686 T1821 oass.SolrIndexSearcher.<init> Opening Searcher@1254a0d[collection1] main
[junit4] 2> 427687 T1821 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 427688 T1821 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 427688 T1821 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 427689 T1821 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 427689 T1821 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 427689 T1821 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 427690 T1821 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 427690 T1821 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 427690 T1821 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 427690 T1821 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 427691 T1822 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1254a0d[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 427692 T1821 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 427693 T1825 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55629/cdh/gy collection:control_collection shard:shard1
[junit4] 2> 427693 T1794 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 427693 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 427693 T1825 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 427695 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 427697 T1827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2611609a name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 427697 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 427698 T1794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 427699 T1825 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 427702 T1794 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 427703 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427703 T1825 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 427703 T1825 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C518 name=collection1 org.apache.solr.core.SolrCore@16176b16 url=https://127.0.0.1:55629/cdh/gy/collection1 node=127.0.0.1:55629_cdh%2Fgy C518_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:55629/cdh/gy, core=collection1, node_name=127.0.0.1:55629_cdh%2Fgy}
[junit4] 2> 427704 T1825 C518 P55629 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55629/cdh/gy/collection1/
[junit4] 2> 427704 T1825 C518 P55629 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 427705 T1825 C518 P55629 oasc.SyncStrategy.syncToMe https://127.0.0.1:55629/cdh/gy/collection1/ has no replicas
[junit4] 2> 427705 T1825 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55629/cdh/gy/collection1/ shard1
[junit4] 2> 427704 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 427705 T1825 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 427708 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427711 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427714 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427816 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 427816 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 427863 T1825 oasc.ZkController.register We are https://127.0.0.1:55629/cdh/gy/collection1/ and leader is https://127.0.0.1:55629/cdh/gy/collection1/
[junit4] 2> 427863 T1825 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55629/cdh/gy
[junit4] 2> 427863 T1825 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 427863 T1825 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 427863 T1825 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 427865 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427865 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427865 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427865 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 427866 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 427866 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:55629/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55629_cdh%2Fgy",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 427870 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 427940 T1794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 427941 T1794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 427943 T1794 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 427944 T1794 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38080
[junit4] 2> 427946 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 427947 T1794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 427947 T1794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003
[junit4] 2> 427948 T1794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/'
[junit4] 2> 427960 T1794 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/solr.xml
[junit4] 2> 427969 T1794 oasc.CoreContainer.<init> New CoreContainer 1279890953
[junit4] 2> 427970 T1794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/]
[junit4] 2> 427970 T1794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 427970 T1794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 427971 T1794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 427971 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 427971 T1794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 427971 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 427972 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 427972 T1794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 427972 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 427972 T1794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 427972 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 427974 T1794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 427974 T1794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 427975 T1794 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 427975 T1794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37566/solr
[junit4] 2> 427975 T1794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 427976 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 427980 T1838 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10d73739 name:ZooKeeperConnection Watcher:127.0.0.1:37566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 427980 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 427983 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 427984 T1840 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f105372 name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 427985 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 427992 T1794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 428996 T1794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38080_cdh%2Fgy
[junit4] 2> 428999 T1794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38080_cdh%2Fgy
[junit4] 2> 429010 T1841 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 429011 T1841 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 429012 T1841 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 429012 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 429013 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 429013 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 429015 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 429016 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38080/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38080_cdh%2Fgy",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 429016 T1819 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 429017 T1819 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 429020 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 429123 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 429123 T1840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 429123 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430012 T1841 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 430013 T1841 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 430014 T1841 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 430014 T1841 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 430015 T1841 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 430016 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/collection1/'
[junit4] 2> 430017 T1841 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 430017 T1841 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 430018 T1841 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 430045 T1841 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 430055 T1841 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 430059 T1841 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 430064 T1841 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 430159 T1841 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 430160 T1841 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 430161 T1841 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 430163 T1841 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 430174 T1841 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 430174 T1841 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 430174 T1841 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/
[junit4] 2> 430175 T1841 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@463b09db
[junit4] 2> 430175 T1841 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1
[junit4] 2> 430175 T1841 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/index/
[junit4] 2> 430175 T1841 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 430176 T1841 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/index
[junit4] 2> 430176 T1841 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=7.9267578125, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=24.099611015611966, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4531112716772604
[junit4] 2> 430186 T1841 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 430186 T1841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 430189 T1841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 430189 T1841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 430189 T1841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 430189 T1841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 430189 T1841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 430190 T1841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 430190 T1841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 430190 T1841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 430190 T1841 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 430190 T1841 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 430191 T1841 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 430191 T1841 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 430191 T1841 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 430191 T1841 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 430192 T1841 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 430192 T1841 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 430192 T1841 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 430194 T1841 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 430195 T1841 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 430195 T1841 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 430195 T1841 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=91.0322265625, floorSegmentMB=1.265625, forceMergeDeletesPctAllowed=17.181881993242474, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.1953125, noCFSRatio=1.0
[junit4] 2> 430196 T1841 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 430196 T1841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 430196 T1841 oass.SolrIndexSearcher.<init> Opening Searcher@62206cdb[collection1] main
[junit4] 2> 430197 T1841 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 430198 T1841 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 430198 T1841 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 430198 T1841 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 430198 T1841 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 430198 T1841 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 430199 T1841 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 430199 T1841 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 430199 T1841 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 430199 T1841 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 430202 T1842 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@62206cdb[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 430203 T1841 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 430204 T1845 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38080/cdh/gy collection:collection1 shard:shard2
[junit4] 2> 430204 T1794 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 430204 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 430205 T1845 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 430209 T1845 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 430211 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430211 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430211 T1845 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 430211 T1845 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 430211 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C519 name=collection1 org.apache.solr.core.SolrCore@335a9a20 url=https://127.0.0.1:38080/cdh/gy/collection1 node=127.0.0.1:38080_cdh%2Fgy C519_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38080/cdh/gy, core=collection1, node_name=127.0.0.1:38080_cdh%2Fgy}
[junit4] 2> 430211 T1845 C519 P38080 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38080/cdh/gy/collection1/
[junit4] 2> 430212 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 430212 T1845 C519 P38080 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 430212 T1845 C519 P38080 oasc.SyncStrategy.syncToMe https://127.0.0.1:38080/cdh/gy/collection1/ has no replicas
[junit4] 2> 430212 T1845 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38080/cdh/gy/collection1/ shard2
[junit4] 2> 430212 T1845 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 430213 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430216 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430217 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430319 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430319 T1840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430319 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430366 T1845 oasc.ZkController.register We are https://127.0.0.1:38080/cdh/gy/collection1/ and leader is https://127.0.0.1:38080/cdh/gy/collection1/
[junit4] 2> 430367 T1845 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38080/cdh/gy
[junit4] 2> 430367 T1845 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 430367 T1845 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 430367 T1845 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 430369 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430369 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430369 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430370 T1845 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 430370 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 430371 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:38080/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38080_cdh%2Fgy",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 430375 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 430423 T1794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 430424 T1794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 430428 T1794 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 430430 T1794 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37692
[junit4] 2> 430432 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 430433 T1794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 430434 T1794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004
[junit4] 2> 430435 T1794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/'
[junit4] 2> 430451 T1794 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/solr.xml
[junit4] 2> 430459 T1794 oasc.CoreContainer.<init> New CoreContainer 847873967
[junit4] 2> 430459 T1794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/]
[junit4] 2> 430460 T1794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 430460 T1794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 430460 T1794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 430460 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 430461 T1794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 430461 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 430461 T1794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 430461 T1794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 430461 T1794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 430463 T1794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 430463 T1794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 430464 T1794 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 430464 T1794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37566/solr
[junit4] 2> 430464 T1794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 430465 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 430467 T1856 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f9bcb5f name:ZooKeeperConnection Watcher:127.0.0.1:37566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 430467 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 430470 T1794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 430473 T1858 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51a7b94e name:ZooKeeperConnection Watcher:127.0.0.1:37566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 430473 T1794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 430477 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430477 T1840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430477 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 430482 T1794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 431486 T1794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37692_cdh%2Fgy
[junit4] 2> 431487 T1794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37692_cdh%2Fgy
[junit4] 2> 431499 T1859 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 431499 T1859 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 431501 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 431501 T1859 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 431501 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 431501 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 431504 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 431504 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37692/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37692_cdh%2Fgy",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 431505 T1819 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 431505 T1819 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 431509 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 431617 T1840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 431617 T1858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 431617 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 431617 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 432501 T1859 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 432502 T1859 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 432503 T1859 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 432503 T1859 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 432504 T1859 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 432505 T1859 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/collection1/'
[junit4] 2> 432506 T1859 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 432506 T1859 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 432507 T1859 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 432540 T1859 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 432552 T1859 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 432553 T1859 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 432557 T1859 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 432648 T1859 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 432649 T1859 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 432650 T1859 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 432652 T1859 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 432664 T1859 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 432664 T1859 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 432664 T1859 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/
[junit4] 2> 432664 T1859 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@463b09db
[junit4] 2> 432665 T1859 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2
[junit4] 2> 432665 T1859 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/index/
[junit4] 2> 432665 T1859 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 432665 T1859 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/index
[junit4] 2> 432666 T1859 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=7.9267578125, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=24.099611015611966, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4531112716772604
[junit4] 2> 432672 T1859 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 432672 T1859 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 432675 T1859 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 432675 T1859 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 432675 T1859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 432675 T1859 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 432676 T1859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 432676 T1859 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 432676 T1859 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 432676 T1859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 432676 T1859 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 432677 T1859 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 432677 T1859 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 432677 T1859 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 432678 T1859 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 432678 T1859 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 432678 T1859 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 432679 T1859 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 432679 T1859 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 432680 T1859 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 432682 T1859 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 432682 T1859 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 432682 T1859 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=91.0322265625, floorSegmentMB=1.265625, forceMergeDeletesPctAllowed=17.181881993242474, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.1953125, noCFSRatio=1.0
[junit4] 2> 432683 T1859 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 432683 T1859 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 432683 T1859 oass.SolrIndexSearcher.<init> Opening Searcher@6d06d39e[collection1] main
[junit4] 2> 432684 T1859 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 432685 T1859 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 432685 T1859 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 432685 T1859 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 432685 T1859 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 432685 T1859 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 432686 T1859 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 432686 T1859 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 432686 T1859 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 432686 T1859 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 432689 T1860 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d06d39e[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 432691 T1859 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 432692 T1863 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:37692/cdh/gy collection:collection1 shard:shard3
[junit4] 2> 432692 T1794 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 432692 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 432693 T1863 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 432697 T1863 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 432699 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432699 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432699 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432699 T1863 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 432699 T1863 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C520 name=collection1 org.apache.solr.core.SolrCore@273751d3 url=https://127.0.0.1:37692/cdh/gy/collection1 node=127.0.0.1:37692_cdh%2Fgy C520_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:37692/cdh/gy, core=collection1, node_name=127.0.0.1:37692_cdh%2Fgy}
[junit4] 2> 432699 T1863 C520 P37692 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:37692/cdh/gy/collection1/
[junit4] 2> 432700 T1863 C520 P37692 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 432700 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 432700 T1863 C520 P37692 oasc.SyncStrategy.syncToMe https://127.0.0.1:37692/cdh/gy/collection1/ has no replicas
[junit4] 2> 432700 T1863 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:37692/cdh/gy/collection1/ shard3
[junit4] 2> 432700 T1863 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 432702 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432704 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432705 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432807 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 432807 T1818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 432807 T1840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 432807 T1858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 432854 T1863 oasc.ZkController.register We are https://127.0.0.1:37692/cdh/gy/collection1/ and leader is https://127.0.0.1:37692/cdh/gy/collection1/
[junit4] 2> 432855 T1863 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37692/cdh/gy
[junit4] 2> 432855 T1863 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 432855 T1863 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 432855 T1863 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 432859 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432859 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432859 T1863 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 432859 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432860 T1819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 432861 T1819 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:37692/cdh/gy",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37692_cdh%2Fgy",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 432862 T1818 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 432916 T1794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 432917 T1794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 432921 T1794 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 432923 T1794 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:36030
[junit4] 2> 432926 T1794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 432926 T1794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 432927 T1794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-D7D1E61AF425929C-001/tempDir-005
[junit4] 2> 432927 T1794 oasc.Sol
[...truncated too long message...]
Dir-006/nodes_used_collection_shard1_replica2/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001/tempDir-006/nodes_used_collection_shard1_replica2/data;done=false>>]
[junit4] 2> 821212 T3766 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001/tempDir-006/nodes_used_collection_shard1_replica2/data
[junit4] 2> 821212 T3766 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001/tempDir-006/nodes_used_collection_shard1_replica2/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001/tempDir-006/nodes_used_collection_shard1_replica2/data/index;done=false>>]
[junit4] 2> 821213 T3766 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001/tempDir-006/nodes_used_collection_shard1_replica2/data/index
[junit4] 2> 821213 T3766 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4] 2> 821214 T3766 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:135)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:290)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:209)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:206)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:206)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:417)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:127)
[junit4] 2> at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:122)
[junit4] 2> ... 7 more
[junit4] 2>
[junit4] 2> 821214 T3766 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 821215 T3766 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-D7D1E61AF425929C-001
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=de_GR, timezone=America/Argentina/San_Luis
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_20-ea (64-bit)/cpus=8,threads=1,free=180284696,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestCodecSupport, PrimUtilsTest, SpellCheckCollatorTest, TermVectorComponentDistributedTest, SolrXmlInZkTest, TestLRUCache, TestSolrXmlPersistence, JSONWriterTest, XsltUpdateRequestHandlerTest, DocumentAnalysisRequestHandlerTest, NumericFieldsTest, TestManagedSynonymFilterFactory, HighlighterConfigTest, AtomicUpdatesTest, TestCollationField, TestInfoStreamLogging, TestQuerySenderListener, FieldAnalysisRequestHandlerTest, SpellCheckComponentTest, SolrIndexConfigTest, TestCSVResponseWriter, TestFreeTextSuggestions, DirectUpdateHandlerTest, TestIBSimilarityFactory, TestRecoveryHdfs, XmlUpdateRequestHandlerTest, SolrPluginUtilsTest, SimplePostToolTest, NotRequiredUniqueKeyTest, TestOmitPositions, PluginInfoTest, SolrCoreCheckLockOnStartupTest, TestMiniSolrCloudCluster, DistanceFunctionTest, IndexSchemaRuntimeFieldTest, TestPhraseSuggestions, TestFuzzyAnalyzedSuggestions, TestQuerySenderNoQuery, TestRecovery, TestAnalyzedSuggestions, CoreMergeIndexesAdminHandlerTest, TestInitQParser, DOMUtilTest, TestFiltering, TestSchemaNameResource, TestSolrQueryParserDefaultOperatorResource, MinimalSchemaTest, TestDistributedGrouping, TestCoreDiscovery, DistribDocExpirationUpdateProcessorTest, AlternateDirectoryTest, DocValuesMultiTest, TestDynamicFieldCollectionResource, TestCloudManagedSchema, TestBinaryField, FunctionTest, TestStressReorder, HdfsBasicDistributedZk2Test, MultiTermTest, TestStressLucene, TestDocumentBuilder, TestExpandComponent, BasicDistributedZk2Test, TestWordDelimiterFilterFactory, TestFastOutputStream, AbstractAnalyticsStatsTest, TestIndexingPerformance, HdfsCollectionsAPIDistributedZkTest, TestFieldCollectionResource, SuggesterTSTTest, TestSolr4Spatial, FieldFacetExtrasTest, TestCustomSort, DeleteInactiveReplicaTest, LegacyHTMLStripCharFilterTest, TestAtomicUpdateErrorCases, TestDFRSimilarityFactory, InfoHandlerTest, TestQueryTypes, TestReplicationHandlerBackup, ConvertedLegacyTest, TestManagedResource, AnalysisErrorHandlingTest, ChaosMonkeySafeLeaderTest, TestDefaultSearchFieldResource, RequestHandlersTest, DefaultValueUpdateProcessorTest, TestSolrCoreProperties, TestLazyCores, TestClassNameShortening, OutputWriterTest, TestBinaryResponseWriter, StandardRequestHandlerTest, DateFieldTest, FullSolrCloudDistribCmdsTest, DirectSolrConnectionTest, FileUtilsTest, TestDynamicFieldResource, BasicDistributedZkTest, CollectionsAPIDistributedZkTest]
[junit4] Completed on J1 in 125.07s, 1 test, 1 error <<< FAILURES!
[...truncated 631 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 error, 1 failure, 36 ignored (15 assumptions)
Total time: 64 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b15) -
Build # 10516 - Still Failing!
Posted by "david.w.smiley@gmail.com" <da...@gmail.com>.
I’m on it.
~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley
On Mon, Jun 9, 2014 at 10:36 AM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10516/
> Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseParallelGC
>
> 1 tests failed.
> FAILED: org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin
> {#4 seed=[3F4202D795A1E146:B3DDDC492CA0D99]}
>
> Error Message:
> Shouldn't match I#2:[-264000 TO -264000-11-20] Q:-264000
>
> Stack Trace:
> java.lang.AssertionError: Shouldn't match I#2:[-264000 TO -264000-11-20]
> Q:-264000
> at
> __randomizedtesting.SeedInfo.seed([3F4202D795A1E146:B3DDDC492CA0D99]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.fail(BaseNonFuzzySpatialOpStrategyTest.java:128)
> at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperation(BaseNonFuzzySpatialOpStrategyTest.java:117)
> at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperationRandomShapes(BaseNonFuzzySpatialOpStrategyTest.java:64)
> at
> org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin(DateNRStrategyTest.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 9391 lines...]
> [junit4] Suite: org.apache.lucene.spatial.prefix.DateNRStrategyTest
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=DateNRStrategyTest -Dtests.method=testWithin
> -Dtests.seed=3F4202D795A1E146 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=ga_IE -Dtests.timezone=America/Argentina/Jujuy
> -Dtests.file.encoding=ISO-8859-1
> [junit4] FAILURE 0.28s J1 | DateNRStrategyTest.testWithin {#4
> seed=[3F4202D795A1E146:B3DDDC492CA0D99]} <<<
> [junit4] > Throwable #1: java.lang.AssertionError: Shouldn't match
> I#2:[-264000 TO -264000-11-20] Q:-264000
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([3F4202D795A1E146:B3DDDC492CA0D99]:0)
> [junit4] > at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.fail(BaseNonFuzzySpatialOpStrategyTest.java:128)
> [junit4] > at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperation(BaseNonFuzzySpatialOpStrategyTest.java:117)
> [junit4] > at
> org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperationRandomShapes(BaseNonFuzzySpatialOpStrategyTest.java:64)
> [junit4] > at
> org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin(DateNRStrategyTest.java:59)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/test/J1/./temp/lucene.spatial.prefix.DateNRStrategyTest-3F4202D795A1E146-001
> [junit4] 2> NOTE: test params are: codec=Lucene40,
> sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ga_IE,
> timezone=America/Argentina/Jujuy
> [junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation
> 1.8.0_20-ea (64-bit)/cpus=8,threads=1,free=150337080,total=224395264
> [junit4] 2> NOTE: All tests run in this JVM: [TestBBoxStrategy,
> DateNRStrategyTest]
> [junit4] Completed on J1 in 8.88s, 42 tests, 1 failure, 10 skipped <<<
> FAILURES!
>
> [...truncated 31 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:543:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2017:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:60:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920:
> There were test failures: 17 suites, 125 tests, 1 failure, 12 ignored (2
> assumptions)
>
> Total time: 30 minutes 1 second
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops
> -XX:+UseParallelGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b15) -
Build # 10516 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10516/
Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin {#4 seed=[3F4202D795A1E146:B3DDDC492CA0D99]}
Error Message:
Shouldn't match I#2:[-264000 TO -264000-11-20] Q:-264000
Stack Trace:
java.lang.AssertionError: Shouldn't match I#2:[-264000 TO -264000-11-20] Q:-264000
at __randomizedtesting.SeedInfo.seed([3F4202D795A1E146:B3DDDC492CA0D99]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.fail(BaseNonFuzzySpatialOpStrategyTest.java:128)
at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperation(BaseNonFuzzySpatialOpStrategyTest.java:117)
at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperationRandomShapes(BaseNonFuzzySpatialOpStrategyTest.java:64)
at org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin(DateNRStrategyTest.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9391 lines...]
[junit4] Suite: org.apache.lucene.spatial.prefix.DateNRStrategyTest
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DateNRStrategyTest -Dtests.method=testWithin -Dtests.seed=3F4202D795A1E146 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ga_IE -Dtests.timezone=America/Argentina/Jujuy -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.28s J1 | DateNRStrategyTest.testWithin {#4 seed=[3F4202D795A1E146:B3DDDC492CA0D99]} <<<
[junit4] > Throwable #1: java.lang.AssertionError: Shouldn't match I#2:[-264000 TO -264000-11-20] Q:-264000
[junit4] > at __randomizedtesting.SeedInfo.seed([3F4202D795A1E146:B3DDDC492CA0D99]:0)
[junit4] > at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.fail(BaseNonFuzzySpatialOpStrategyTest.java:128)
[junit4] > at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperation(BaseNonFuzzySpatialOpStrategyTest.java:117)
[junit4] > at org.apache.lucene.spatial.prefix.BaseNonFuzzySpatialOpStrategyTest.testOperationRandomShapes(BaseNonFuzzySpatialOpStrategyTest.java:64)
[junit4] > at org.apache.lucene.spatial.prefix.DateNRStrategyTest.testWithin(DateNRStrategyTest.java:59)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/test/J1/./temp/lucene.spatial.prefix.DateNRStrategyTest-3F4202D795A1E146-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ga_IE, timezone=America/Argentina/Jujuy
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_20-ea (64-bit)/cpus=8,threads=1,free=150337080,total=224395264
[junit4] 2> NOTE: All tests run in this JVM: [TestBBoxStrategy, DateNRStrategyTest]
[junit4] Completed on J1 in 8.88s, 42 tests, 1 failure, 10 skipped <<< FAILURES!
[...truncated 31 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:543: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2017: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 17 suites, 125 tests, 1 failure, 12 ignored (2 assumptions)
Total time: 30 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_05) - Build #
10515 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10515/
Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard2 is not consistent. Got 513 from http://127.0.0.1:49972/collection1lastClient and got 515 from http://127.0.0.1:54538/collection1
Stack Trace:
java.lang.AssertionError: shard2 is not consistent. Got 513 from http://127.0.0.1:49972/collection1lastClient and got 515 from http://127.0.0.1:54538/collection1
at __randomizedtesting.SeedInfo.seed([96A231B0F93D96E2:1744BFA88E62F6DE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([96A231B0F93D96E2:1744BFA88E62F6DE]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:317)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:650)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1141)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:236)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10678 lines...]
[junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/init-core-data-001
[junit4] 2> 75250 T210 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 75251 T210 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 75254 T210 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 75254 T210 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 75255 T211 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 75355 T210 oasc.ZkTestServer.run start zk server on port:47477
[junit4] 2> 75356 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75469 T217 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6afce936 name:ZooKeeperConnection Watcher:127.0.0.1:47477 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75469 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75469 T210 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 75474 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75476 T219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c7cdc26 name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75476 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75477 T210 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 75480 T210 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 75483 T210 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 75489 T210 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 75492 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 75495 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 75503 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 75504 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 75506 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 75507 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 75509 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 75509 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 75511 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 75512 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 75513 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 75514 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 75515 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 75516 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 75518 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 75518 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 75520 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 75520 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 75522 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 75523 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 75524 T210 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 75525 T210 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 75531 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75536 T221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7de31d54 name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75537 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75851 T210 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 76094 T210 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 76101 T210 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:53725
[junit4] 2> 76105 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 76105 T210 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 76106 T210 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002
[junit4] 2> 76106 T210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/'
[junit4] 2> 76123 T210 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/solr.xml
[junit4] 2> 76154 T210 oasc.CoreContainer.<init> New CoreContainer 352180056
[junit4] 2> 76154 T210 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/]
[junit4] 2> 76155 T210 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 76155 T210 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 76156 T210 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 76156 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 76156 T210 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 76157 T210 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 76157 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 76157 T210 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 76157 T210 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 76160 T210 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 76161 T210 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 76161 T210 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 76161 T210 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47477/solr
[junit4] 2> 76162 T210 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 76163 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 76173 T232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74a0c31d name:ZooKeeperConnection Watcher:127.0.0.1:47477 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 76174 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 76185 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 76187 T234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4847bd07 name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 76188 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 76189 T210 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 76193 T210 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 76196 T210 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 76203 T210 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 76206 T210 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 76209 T210 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 76219 T210 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53725_
[junit4] 2> 76221 T210 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53725_
[junit4] 2> 76232 T210 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 76234 T210 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 76240 T210 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:53725_
[junit4] 2> 76241 T210 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 76243 T210 oasc.Overseer.start Overseer (id=91902198886825988-127.0.0.1:53725_-n_0000000000) starting
[junit4] 2> 76248 T210 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 76259 T236 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 76259 T210 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 76266 T210 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 76269 T210 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 76274 T237 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 76274 T237 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 76278 T237 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 76283 T235 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 76285 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 76285 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53725",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53725_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 76285 T235 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 76286 T235 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 76288 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76289 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 77278 T237 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 77279 T237 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 77279 T237 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 77279 T237 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 77280 T237 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 77280 T237 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/collection1/'
[junit4] 2> 77281 T237 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 77281 T237 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 77281 T237 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 77299 T237 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 77307 T237 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 77308 T237 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 77312 T237 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 77394 T237 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 77398 T237 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 77399 T237 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 77408 T237 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 77411 T237 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 77413 T237 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 77414 T237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 77414 T237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 77414 T237 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 77415 T237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 77415 T237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 77415 T237 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 77415 T237 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 77416 T237 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/control/data/
[junit4] 2> 77416 T237 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@40629871
[junit4] 2> 77416 T237 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/control/data
[junit4] 2> 77416 T237 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/control/data/index/
[junit4] 2> 77417 T237 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 77417 T237 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/control/data/index
[junit4] 2> 77417 T237 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=0.505859375, floorSegmentMB=1.1787109375, forceMergeDeletesPctAllowed=23.873940951367725, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8562724387992168
[junit4] 2> 77418 T237 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@65430b48 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7ac048c0),segFN=segments_1,generation=1}
[junit4] 2> 77418 T237 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77420 T237 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 77420 T237 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 77421 T237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 77421 T237 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 77421 T237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 77421 T237 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 77421 T237 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 77422 T237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 77422 T237 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 77422 T237 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 77422 T237 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 77423 T237 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 77423 T237 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 77423 T237 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 77423 T237 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 77423 T237 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 77424 T237 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 77425 T237 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 77427 T237 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 77427 T237 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 77427 T237 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=91.916015625, floorSegmentMB=1.05859375, forceMergeDeletesPctAllowed=11.790924324113863, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 77428 T237 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@65430b48 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7ac048c0),segFN=segments_1,generation=1}
[junit4] 2> 77428 T237 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77428 T237 oass.SolrIndexSearcher.<init> Opening Searcher@565e2c61[collection1] main
[junit4] 2> 77428 T237 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 77429 T237 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 77429 T237 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 77430 T237 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 77430 T237 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 77430 T237 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 77430 T237 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 77431 T237 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 77431 T237 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 77431 T237 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 77434 T237 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 77435 T238 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@565e2c61[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 77435 T241 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:53725 collection:control_collection shard:shard1
[junit4] 2> 77436 T210 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 77436 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 77436 T241 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 77440 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77442 T241 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 77442 T243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f5c0da6 name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77443 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77444 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77444 T241 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 77444 T241 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@1dc6e8ae url=https://127.0.0.1:53725/collection1 node=127.0.0.1:53725_ C25_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:53725, core=collection1, node_name=127.0.0.1:53725_}
[junit4] 2> 77444 T241 C25 P53725 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:53725/collection1/
[junit4] 2> 77445 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77444 T210 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 77445 T241 C25 P53725 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 77445 T241 C25 P53725 oasc.SyncStrategy.syncToMe https://127.0.0.1:53725/collection1/ has no replicas
[junit4] 2> 77446 T241 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:53725/collection1/ shard1
[junit4] 2> 77446 T241 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 77448 T210 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 77450 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77451 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77453 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77555 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 77555 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 77602 T241 oasc.ZkController.register We are https://127.0.0.1:53725/collection1/ and leader is https://127.0.0.1:53725/collection1/
[junit4] 2> 77603 T241 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:53725
[junit4] 2> 77603 T241 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77603 T241 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 77603 T241 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77604 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77605 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77605 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77605 T241 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77606 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77607 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:53725",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53725_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 77608 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77678 T210 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 77679 T210 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 77684 T210 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 77687 T210 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:34253
[junit4] 2> 77689 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 77689 T210 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 77689 T210 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003
[junit4] 2> 77690 T210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/'
[junit4] 2> 77700 T210 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/solr.xml
[junit4] 2> 77712 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 77712 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 77715 T210 oasc.CoreContainer.<init> New CoreContainer 1956401431
[junit4] 2> 77715 T210 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/]
[junit4] 2> 77716 T210 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 77716 T210 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 77716 T210 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 77716 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77716 T210 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77717 T210 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77717 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77717 T210 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77717 T210 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77720 T210 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77721 T210 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77721 T210 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77721 T210 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47477/solr
[junit4] 2> 77721 T210 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77722 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77724 T254 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f4df2b name:ZooKeeperConnection Watcher:127.0.0.1:47477 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77724 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77731 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77737 T256 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@116313b5 name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77737 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77743 T210 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 78745 T210 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34253_
[junit4] 2> 78746 T210 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34253_
[junit4] 2> 78754 T257 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 78755 T257 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78756 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78757 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78757 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78756 T257 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78759 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78759 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34253",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34253_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78759 T235 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 78760 T235 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 78763 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78865 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 78865 T256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 78865 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 79757 T257 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79758 T257 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79759 T257 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79760 T257 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79761 T257 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79762 T257 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/collection1/'
[junit4] 2> 79762 T257 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 79762 T257 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 79763 T257 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 79797 T257 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 79807 T257 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79808 T257 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79813 T257 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79955 T257 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 79960 T257 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79961 T257 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79971 T257 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 79976 T257 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 79980 T257 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 79981 T257 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 79988 T257 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 79988 T257 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 79990 T257 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 79990 T257 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 79990 T257 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 79991 T257 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79991 T257 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty1/
[junit4] 2> 79992 T257 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@40629871
[junit4] 2> 79993 T257 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty1
[junit4] 2> 79993 T257 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty1/index/
[junit4] 2> 79993 T257 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 79993 T257 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty1/index
[junit4] 2> 79994 T257 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=0.505859375, floorSegmentMB=1.1787109375, forceMergeDeletesPctAllowed=23.873940951367725, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8562724387992168
[junit4] 2> 79995 T257 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@65723a0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5f63851d),segFN=segments_1,generation=1}
[junit4] 2> 79996 T257 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 80002 T257 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 80003 T257 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 80003 T257 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 80003 T257 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 80004 T257 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 80004 T257 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 80004 T257 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 80005 T257 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 80005 T257 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 80006 T257 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 80006 T257 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 80006 T257 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 80007 T257 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 80007 T257 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 80007 T257 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 80008 T257 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 80008 T257 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 80010 T257 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 80015 T257 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 80016 T257 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 80017 T257 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=91.916015625, floorSegmentMB=1.05859375, forceMergeDeletesPctAllowed=11.790924324113863, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 80018 T257 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@65723a0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5f63851d),segFN=segments_1,generation=1}
[junit4] 2> 80018 T257 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 80018 T257 oass.SolrIndexSearcher.<init> Opening Searcher@7b9e894e[collection1] main
[junit4] 2> 80018 T257 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 80020 T257 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 80020 T257 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 80020 T257 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 80021 T257 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 80021 T257 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 80022 T257 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 80022 T257 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 80022 T257 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 80022 T257 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 80025 T258 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b9e894e[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 80027 T257 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 80027 T261 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:34253 collection:collection1 shard:shard1
[junit4] 2> 80028 T210 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 80028 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 80029 T261 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 80042 T261 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 80048 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80048 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80048 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80049 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80050 T261 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 80050 T261 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@45edcdac url=https://127.0.0.1:34253/collection1 node=127.0.0.1:34253_ C26_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:34253, core=collection1, node_name=127.0.0.1:34253_}
[junit4] 2> 80050 T261 C26 P34253 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:34253/collection1/
[junit4] 2> 80050 T261 C26 P34253 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 80051 T261 C26 P34253 oasc.SyncStrategy.syncToMe https://127.0.0.1:34253/collection1/ has no replicas
[junit4] 2> 80051 T261 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:34253/collection1/ shard1
[junit4] 2> 80051 T261 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 80052 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80065 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80167 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80167 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80167 T256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80175 T261 oasc.ZkController.register We are https://127.0.0.1:34253/collection1/ and leader is https://127.0.0.1:34253/collection1/
[junit4] 2> 80175 T261 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34253
[junit4] 2> 80176 T261 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 80176 T261 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 80176 T261 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 80178 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80178 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80178 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80179 T261 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80180 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80180 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:34253",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34253_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 80182 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80284 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80284 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80284 T256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 80300 T210 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 80301 T210 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 80308 T210 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 80311 T210 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:49240
[junit4] 2> 80313 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 80314 T210 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 80314 T210 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004
[junit4] 2> 80315 T210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/'
[junit4] 2> 80326 T210 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/solr.xml
[junit4] 2> 80340 T210 oasc.CoreContainer.<init> New CoreContainer 375141446
[junit4] 2> 80340 T210 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/]
[junit4] 2> 80341 T210 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 80341 T210 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 80341 T210 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 80342 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 80342 T210 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 80342 T210 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 80343 T210 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 80343 T210 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 80343 T210 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 80348 T210 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 80349 T210 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 80349 T210 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 80349 T210 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47477/solr
[junit4] 2> 80349 T210 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 80350 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80354 T272 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b691793 name:ZooKeeperConnection Watcher:127.0.0.1:47477 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80363 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80366 T210 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80368 T274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68a779de name:ZooKeeperConnection Watcher:127.0.0.1:47477/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80369 T210 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80380 T210 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 81386 T210 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49240_
[junit4] 2> 81387 T210 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49240_
[junit4] 2> 81394 T275 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 81394 T275 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 81395 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81395 T275 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 81395 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81396 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81396 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 81397 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49240",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49240_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 81397 T235 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 81397 T235 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 81399 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81501 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 81501 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 81501 T274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 81501 T256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 82395 T275 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 82396 T275 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 82396 T275 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 82397 T275 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 82397 T275 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 82398 T275 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/collection1/'
[junit4] 2> 82398 T275 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 82399 T275 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 82399 T275 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 82426 T275 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 82437 T275 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 82439 T275 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 82445 T275 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 82574 T275 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 82579 T275 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 82581 T275 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 82591 T275 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 82594 T275 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 82597 T275 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 82598 T275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 82599 T275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 82599 T275 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 82600 T275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 82600 T275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 82601 T275 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 82601 T275 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 82601 T275 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty2/
[junit4] 2> 82602 T275 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@40629871
[junit4] 2> 82603 T275 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty2
[junit4] 2> 82603 T275 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty2/index/
[junit4] 2> 82603 T275 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 82604 T275 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-001/jetty2/index
[junit4] 2> 82605 T275 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=0.505859375, floorSegmentMB=1.1787109375, forceMergeDeletesPctAllowed=23.873940951367725, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8562724387992168
[junit4] 2> 82606 T275 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2b1b7d5a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7a9ffde),segFN=segments_1,generation=1}
[junit4] 2> 82606 T275 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 82610 T275 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 82610 T275 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 82611 T275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 82611 T275 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 82611 T275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 82611 T275 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 82612 T275 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 82612 T275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 82612 T275 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 82613 T275 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 82613 T275 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 82613 T275 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 82614 T275 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 82614 T275 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 82615 T275 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 82615 T275 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 82615 T275 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 82618 T275 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 82620 T275 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 82620 T275 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 82621 T275 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=91.916015625, floorSegmentMB=1.05859375, forceMergeDeletesPctAllowed=11.790924324113863, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 82622 T275 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2b1b7d5a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7a9ffde),segFN=segments_1,generation=1}
[junit4] 2> 82622 T275 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 82622 T275 oass.SolrIndexSearcher.<init> Opening Searcher@6f96c3c8[collection1] main
[junit4] 2> 82622 T275 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 82623 T275 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 82624 T275 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 82624 T275 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 82624 T275 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 82625 T275 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 82625 T275 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 82626 T275 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 82626 T275 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 82626 T275 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 82631 T276 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f96c3c8[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 82631 T275 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 82633 T279 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:49240 collection:collection1 shard:shard1
[junit4] 2> 82633 T210 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 82633 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 82635 T279 oasc.ZkController.register We are https://127.0.0.1:49240/collection1/ and leader is https://127.0.0.1:34253/collection1/
[junit4] 2> 82635 T279 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:49240
[junit4] 2> 82636 T279 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 82636 T279 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C27 name=collection1 org.apache.solr.core.SolrCore@7f5bf2ae url=https://127.0.0.1:49240/collection1 node=127.0.0.1:49240_ C27_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:49240, core=collection1, node_name=127.0.0.1:49240_}
[junit4] 2> 82653 T280 C27 P49240 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 82653 T279 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 82654 T280 C27 P49240 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 82654 T280 C27 P49240 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is https://127.0.0.1:34253/collection1/ and I am https://127.0.0.1:49240/collection1/
[junit4] 2> 82658 T280 C27 P49240 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 82659 T280 C27 P49240 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 82674 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 82675 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 82675 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 82676 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 82676 T235 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"https://127.0.0.1:49240",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49240_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 82678 T280 C27 P49240 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to https://127.0.0.1:34253; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A49240_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> 82684 T234 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 82796 T256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 82796 T274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 82796 T243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 82796 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 82833 T252 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 82834 T252 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 82835 T252 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 82835 T252 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:49240_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"state":"recovering","base_url":"https://127.0.0.1:49240","core":"collection1","node_name":"127.0.0.1:49240_"}
[junit4] 2> 82836 T252 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 82836 T252 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:49240_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 82987 T210 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 82988 T210 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 82993 T210 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 82996 T210 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51491
[junit4] 2> 82998 T210 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 82999 T210 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 82999 T210 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-005
[junit4] 2> 82999 T210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-96A231B0F93D96E2-001/tempDir-005/'
[junit4] 2> 83018 T210 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/
[...truncated too long message...]
orkspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001/tempDir-001/jetty10;done=false>>]
[junit4] 2> 272707 T1206 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001/tempDir-001/jetty10
[junit4] 2> 272707 T1206 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001/tempDir-001/jetty10/index.20140609141956277 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001/tempDir-001/jetty10/index.20140609141956277;done=false>>]
[junit4] 2> 272707 T1206 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001/tempDir-001/jetty10/index.20140609141956277
[junit4] 2> 272708 T1206 oasc.LeaderElector$ElectionWatcher.process WARN java.lang.RuntimeException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1273)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1224)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1220)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:190)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:312)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:309)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:309)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1264)
[junit4] 2> ... 24 more
[junit4] 2>
[junit4] 2> 272708 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 10)
[junit4] 2> 272708 T1206 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 272709 T1206 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 272709 T1206 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.method=testDistribSearch -Dtests.seed=96A231B0F93D96E2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=Europe/Warsaw -Dtests.file.encoding=UTF-8
[junit4] FAILURE 154s J1 | ChaosMonkeySafeLeaderTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: shard2 is not consistent. Got 513 from http://127.0.0.1:49972/collection1lastClient and got 515 from http://127.0.0.1:54538/collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([96A231B0F93D96E2:1744BFA88E62F6DE]:0)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
[junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 272731 T358 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-96A231B0F93D96E2-001
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=DefaultSimilarity, locale=lt, timezone=Europe/Warsaw
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_05 (64-bit)/cpus=8,threads=1,free=57479552,total=199266304
[junit4] 2> NOTE: All tests run in this JVM: [ZkSolrClientTest, SSLMigrationTest, AtomicUpdatesTest, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J1 in 154.31s, 1 test, 1 failure <<< FAILURES!
[...truncated 1124 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1692 tests, 1 error, 1 failure, 36 ignored (15 assumptions)
Total time: 64 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60) - Build #
10514 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10514/
Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([7733E4D74C9F7CFD:F6D56ACF3BC01CC1]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:317)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:650)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1141)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:236)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11215 lines...]
[junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/init-core-data-001
[junit4] 2> 926319 T4495 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 926320 T4495 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 926325 T4495 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 926325 T4495 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 926326 T4496 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 926426 T4495 oasc.ZkTestServer.run start zk server on port:51029
[junit4] 2> 926427 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 926430 T4502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@717a960e name:ZooKeeperConnection Watcher:127.0.0.1:51029 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926430 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 926431 T4495 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 926434 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 926435 T4504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e987443 name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926435 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 926436 T4495 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 926437 T4495 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 926439 T4495 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 926440 T4495 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 926441 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 926442 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 926444 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 926445 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 926447 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 926447 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 926449 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 926449 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 926451 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 926451 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 926453 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 926453 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 926454 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 926455 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 926457 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 926457 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 926459 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 926459 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 926461 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 926461 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 926462 T4495 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 926463 T4495 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 926464 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 926466 T4506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cf7774a name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926466 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 926676 T4495 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 926681 T4495 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 926703 T4495 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:40524
[junit4] 2> 926706 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 926706 T4495 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 926707 T4495 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002
[junit4] 2> 926707 T4495 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/'
[junit4] 2> 926726 T4495 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/solr.xml
[junit4] 2> 926759 T4495 oasc.CoreContainer.<init> New CoreContainer 1969963656
[junit4] 2> 926759 T4495 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/]
[junit4] 2> 926760 T4495 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 926760 T4495 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 926760 T4495 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 926761 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 926761 T4495 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 926761 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 926761 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 926761 T4495 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 926762 T4495 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 926764 T4495 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 926764 T4495 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 926764 T4495 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 926764 T4495 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51029/solr
[junit4] 2> 926765 T4495 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 926765 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 926767 T4517 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@679d97ce name:ZooKeeperConnection Watcher:127.0.0.1:51029 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926767 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 926770 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 926772 T4519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e50266b name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926773 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 926775 T4495 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 926780 T4495 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 926784 T4495 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 926789 T4495 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 926793 T4495 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 926798 T4495 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 926808 T4495 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40524_
[junit4] 2> 926810 T4495 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40524_
[junit4] 2> 926815 T4495 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 926818 T4495 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 926823 T4495 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40524_
[junit4] 2> 926824 T4495 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 926827 T4495 oasc.Overseer.start Overseer (id=91901731441278980-127.0.0.1:40524_-n_0000000000) starting
[junit4] 2> 926831 T4495 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 926846 T4521 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 926847 T4495 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 926851 T4495 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 926852 T4495 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 926854 T4520 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 926860 T4522 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 926860 T4522 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 926861 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 926861 T4522 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 926863 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 926864 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40524",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40524_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 926865 T4520 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 926865 T4520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 926869 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 926869 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 927862 T4522 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 927862 T4522 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 927863 T4522 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 927864 T4522 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 927865 T4522 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 927865 T4522 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/collection1/'
[junit4] 2> 927866 T4522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 927867 T4522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 927867 T4522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 927919 T4522 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 927944 T4522 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 927945 T4522 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 927952 T4522 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 928173 T4522 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 928177 T4522 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 928179 T4522 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 928191 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 928195 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 928200 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 928201 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 928202 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 928202 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 928203 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 928204 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 928204 T4522 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 928204 T4522 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 928205 T4522 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/control/data/
[junit4] 2> 928205 T4522 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6835755d
[junit4] 2> 928207 T4522 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/control/data
[junit4] 2> 928207 T4522 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/control/data/index/
[junit4] 2> 928208 T4522 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 928208 T4522 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/control/data/index
[junit4] 2> 928209 T4522 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 928210 T4522 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@50197cb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@763b6ff4),segFN=segments_1,generation=1}
[junit4] 2> 928210 T4522 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 928216 T4522 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 928217 T4522 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 928217 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 928218 T4522 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 928218 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 928218 T4522 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 928219 T4522 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 928220 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 928220 T4522 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 928221 T4522 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 928221 T4522 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 928222 T4522 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 928222 T4522 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 928223 T4522 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 928223 T4522 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 928224 T4522 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 928225 T4522 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 928229 T4522 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 928233 T4522 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 928233 T4522 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 928234 T4522 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 928235 T4522 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@50197cb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@763b6ff4),segFN=segments_1,generation=1}
[junit4] 2> 928236 T4522 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 928236 T4522 oass.SolrIndexSearcher.<init> Opening Searcher@2a025739[collection1] main
[junit4] 2> 928236 T4522 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 928238 T4522 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 928238 T4522 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 928239 T4522 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 928239 T4522 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 928239 T4522 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 928240 T4522 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 928240 T4522 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 928240 T4522 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 928241 T4522 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 928242 T4523 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a025739[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 928243 T4522 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 928244 T4526 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:40524 collection:control_collection shard:shard1
[junit4] 2> 928244 T4495 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 928245 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 928245 T4526 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 928248 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928249 T4528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@607b4c05 name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928249 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928250 T4526 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 928250 T4495 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 928251 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928252 T4526 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 928252 T4526 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C777 name=collection1 org.apache.solr.core.SolrCore@3c8e6566 url=https://127.0.0.1:40524/collection1 node=127.0.0.1:40524_ C777_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:40524, core=collection1, node_name=127.0.0.1:40524_}
[junit4] 2> 928253 T4526 C777 P40524 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:40524/collection1/
[junit4] 2> 928253 T4526 C777 P40524 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 928253 T4526 C777 P40524 oasc.SyncStrategy.syncToMe https://127.0.0.1:40524/collection1/ has no replicas
[junit4] 2> 928254 T4526 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:40524/collection1/ shard1
[junit4] 2> 928254 T4526 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 928255 T4495 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 928255 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 928260 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928262 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928267 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928370 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 928370 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 928413 T4526 oasc.ZkController.register We are https://127.0.0.1:40524/collection1/ and leader is https://127.0.0.1:40524/collection1/
[junit4] 2> 928413 T4526 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:40524
[junit4] 2> 928413 T4526 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 928414 T4526 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 928414 T4526 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 928415 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928415 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928415 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928415 T4526 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 928416 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 928417 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:40524",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40524_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 928418 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 928479 T4495 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 928480 T4495 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 928485 T4495 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 928487 T4495 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56441
[junit4] 2> 928490 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 928490 T4495 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 928490 T4495 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003
[junit4] 2> 928490 T4495 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/'
[junit4] 2> 928506 T4495 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/solr.xml
[junit4] 2> 928520 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 928520 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 928533 T4495 oasc.CoreContainer.<init> New CoreContainer 1915715664
[junit4] 2> 928534 T4495 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/]
[junit4] 2> 928534 T4495 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 928535 T4495 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 928535 T4495 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 928535 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 928535 T4495 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 928535 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 928536 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 928536 T4495 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 928536 T4495 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 928538 T4495 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 928538 T4495 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 928539 T4495 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 928539 T4495 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51029/solr
[junit4] 2> 928539 T4495 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 928540 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928541 T4539 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7cfdeead name:ZooKeeperConnection Watcher:127.0.0.1:51029 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928541 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928543 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928544 T4541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64f22911 name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928545 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928553 T4495 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 929555 T4495 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56441_
[junit4] 2> 929556 T4495 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56441_
[junit4] 2> 929578 T4542 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 929579 T4542 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 929580 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929580 T4542 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 929580 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929581 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929583 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 929583 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:56441",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56441_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 929584 T4520 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 929584 T4520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 929587 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929690 T4541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 929690 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 929690 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 930581 T4542 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 930581 T4542 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 930582 T4542 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 930582 T4542 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 930584 T4542 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 930584 T4542 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/collection1/'
[junit4] 2> 930586 T4542 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 930586 T4542 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 930587 T4542 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 930644 T4542 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 930667 T4542 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 930669 T4542 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 930675 T4542 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 930863 T4542 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 930868 T4542 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 930870 T4542 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 930881 T4542 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930885 T4542 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930888 T4542 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 930889 T4542 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 930890 T4542 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 930890 T4542 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 930891 T4542 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 930892 T4542 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 930892 T4542 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 930892 T4542 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 930893 T4542 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty1/
[junit4] 2> 930893 T4542 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6835755d
[junit4] 2> 930894 T4542 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty1
[junit4] 2> 930894 T4542 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty1/index/
[junit4] 2> 930895 T4542 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 930895 T4542 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty1/index
[junit4] 2> 930895 T4542 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 930896 T4542 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@36b1afa3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@157824ca),segFN=segments_1,generation=1}
[junit4] 2> 930896 T4542 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 930902 T4542 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 930902 T4542 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 930903 T4542 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 930903 T4542 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 930903 T4542 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 930903 T4542 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 930904 T4542 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 930904 T4542 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 930905 T4542 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 930905 T4542 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 930906 T4542 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 930906 T4542 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 930906 T4542 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 930907 T4542 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 930907 T4542 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 930908 T4542 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 930909 T4542 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 930912 T4542 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 930915 T4542 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 930915 T4542 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 930916 T4542 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 930917 T4542 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@36b1afa3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@157824ca),segFN=segments_1,generation=1}
[junit4] 2> 930917 T4542 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 930918 T4542 oass.SolrIndexSearcher.<init> Opening Searcher@309504a9[collection1] main
[junit4] 2> 930918 T4542 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 930920 T4542 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 930920 T4542 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 930921 T4542 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 930921 T4542 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 930921 T4542 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 930921 T4542 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 930921 T4542 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 930922 T4542 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 930922 T4542 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 930926 T4543 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@309504a9[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 930930 T4542 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 930931 T4546 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:56441 collection:collection1 shard:shard1
[junit4] 2> 930932 T4495 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 930932 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 930932 T4546 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 930938 T4546 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 930940 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930940 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930940 T4546 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 930940 T4546 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 930940 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C778 name=collection1 org.apache.solr.core.SolrCore@609ba084 url=https://127.0.0.1:56441/collection1 node=127.0.0.1:56441_ C778_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:56441, core=collection1, node_name=127.0.0.1:56441_}
[junit4] 2> 930940 T4546 C778 P56441 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:56441/collection1/
[junit4] 2> 930941 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 930941 T4546 C778 P56441 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 930941 T4546 C778 P56441 oasc.SyncStrategy.syncToMe https://127.0.0.1:56441/collection1/ has no replicas
[junit4] 2> 930941 T4546 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:56441/collection1/ shard1
[junit4] 2> 930941 T4546 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 930942 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930944 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930949 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931050 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931051 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931051 T4541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931095 T4546 oasc.ZkController.register We are https://127.0.0.1:56441/collection1/ and leader is https://127.0.0.1:56441/collection1/
[junit4] 2> 931095 T4546 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:56441
[junit4] 2> 931095 T4546 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 931096 T4546 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 931096 T4546 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 931097 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931097 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931097 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931098 T4546 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 931098 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 931099 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:56441",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56441_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 931100 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931149 T4495 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 931149 T4495 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 931153 T4495 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 931154 T4495 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38661
[junit4] 2> 931156 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 931157 T4495 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 931157 T4495 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004
[junit4] 2> 931157 T4495 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/'
[junit4] 2> 931172 T4495 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/solr.xml
[junit4] 2> 931209 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931209 T4541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931209 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931221 T4495 oasc.CoreContainer.<init> New CoreContainer 952781019
[junit4] 2> 931222 T4495 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/]
[junit4] 2> 931223 T4495 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 931223 T4495 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 931223 T4495 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 931223 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 931223 T4495 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 931224 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 931224 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 931224 T4495 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 931224 T4495 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 931227 T4495 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 931227 T4495 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 931227 T4495 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 931228 T4495 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51029/solr
[junit4] 2> 931228 T4495 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 931229 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 931230 T4557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cfa11c5 name:ZooKeeperConnection Watcher:127.0.0.1:51029 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 931230 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 931232 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 931233 T4559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c7f0d4d name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 931233 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 931237 T4495 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 932239 T4495 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38661_
[junit4] 2> 932241 T4495 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38661_
[junit4] 2> 932256 T4560 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 932256 T4560 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 932258 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 932258 T4560 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 932258 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 932259 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 932260 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 932261 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38661",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38661_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 932262 T4520 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 932262 T4520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 932265 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 932367 T4559 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932368 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932368 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932368 T4541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933259 T4560 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 933259 T4560 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 933260 T4560 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 933261 T4560 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 933262 T4560 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 933262 T4560 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/collection1/'
[junit4] 2> 933264 T4560 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 933264 T4560 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 933265 T4560 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 933322 T4560 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 933344 T4560 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 933345 T4560 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 933350 T4560 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 933528 T4560 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 933532 T4560 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 933534 T4560 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 933541 T4560 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 933543 T4560 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 933545 T4560 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 933546 T4560 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 933546 T4560 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 933546 T4560 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 933547 T4560 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 933547 T4560 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 933547 T4560 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 933547 T4560 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 933548 T4560 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty2/
[junit4] 2> 933548 T4560 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6835755d
[junit4] 2> 933549 T4560 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty2
[junit4] 2> 933549 T4560 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty2/index/
[junit4] 2> 933549 T4560 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 933549 T4560 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty2/index
[junit4] 2> 933550 T4560 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 933550 T4560 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5354654 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7f647b25),segFN=segments_1,generation=1}
[junit4] 2> 933550 T4560 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 933555 T4560 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 933556 T4560 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 933556 T4560 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 933556 T4560 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 933557 T4560 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 933557 T4560 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 933557 T4560 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 933557 T4560 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 933558 T4560 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 933558 T4560 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 933559 T4560 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 933559 T4560 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 933559 T4560 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 933559 T4560 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 933560 T4560 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 933560 T4560 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 933561 T4560 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 933563 T4560 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 933565 T4560 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 933565 T4560 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 933565 T4560 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 933566 T4560 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5354654 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7f647b25),segFN=segments_1,generation=1}
[junit4] 2> 933566 T4560 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 933566 T4560 oass.SolrIndexSearcher.<init> Opening Searcher@170d6fb1[collection1] main
[junit4] 2> 933566 T4560 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 933567 T4560 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 933567 T4560 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 933568 T4560 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 933568 T4560 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 933568 T4560 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 933569 T4560 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 933569 T4560 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 933569 T4560 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 933569 T4560 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 933571 T4561 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@170d6fb1[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 933573 T4560 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 933574 T4564 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38661 collection:collection1 shard:shard1
[junit4] 2> 933574 T4495 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 933574 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 933576 T4564 oasc.ZkController.register We are https://127.0.0.1:38661/collection1/ and leader is https://127.0.0.1:56441/collection1/
[junit4] 2> 933576 T4564 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38661
[junit4] 2> 933576 T4564 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 933576 T4564 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C779 name=collection1 org.apache.solr.core.SolrCore@633414aa url=https://127.0.0.1:38661/collection1 node=127.0.0.1:38661_ C779_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38661, core=collection1, node_name=127.0.0.1:38661_}
[junit4] 2> 933577 T4565 C779 P38661 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 933577 T4564 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933578 T4565 C779 P38661 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 933578 T4565 C779 P38661 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is https://127.0.0.1:56441/collection1/ and I am https://127.0.0.1:38661/collection1/
[junit4] 2> 933578 T4565 C779 P38661 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 933579 T4565 C779 P38661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 933580 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933580 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933580 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933581 T4520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933581 T4565 C779 P38661 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to https://127.0.0.1:56441; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A38661_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> 933582 T4520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"https://127.0.0.1:38661",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38661_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 933585 T4519 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933652 T4535 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 933653 T4535 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933653 T4535 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 933654 T4535 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:38661_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"state":"down","base_url":"https://127.0.0.1:38661","core":"collection1","node_name":"127.0.0.1:38661_"}
[junit4] 2> 933691 T4528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933691 T4559 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933691 T4541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933691 T4519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933802 T4495 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 933802 T4495 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 933806 T4495 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 933807 T4495 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:48723
[junit4] 2> 933809 T4495 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 933809 T4495 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 933810 T4495 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-005
[junit4] 2> 933810 T4495 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-005/'
[junit4] 2> 933824 T4495 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-005/solr.xml
[junit4] 2> 933847 T4495 oasc.CoreContainer.<init> New CoreContainer 1418664475
[junit4] 2> 933847 T4495 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-005/]
[junit4] 2> 933848 T4495 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 933848 T4495 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 933848 T4495 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 933848 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 933849 T4495 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 933849 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 933849 T4495 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 933849 T4495 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 933850 T4495 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 933852 T4495 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 933852 T4495 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 933852 T4495 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 933852 T4495 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51029/solr
[junit4] 2> 933853 T4495 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 933853 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 933855 T4577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@148c6c97 name:ZooKeeperConnection Watcher:127.0.0.1:51029 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 933855 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 933858 T4495 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 933860 T4579 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e2b2a9c name:ZooKeeperConnection Watcher:127.0.0.1:51029/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 933860 T4495 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 933873 T4495 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 934654 T4535 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:38661_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"state":"recovering","base_url":"https://127.0.0.1:38661","core":"collection1","node_name":"127.0.0.1:38661_"}
[junit4] 2> 934655 T4535 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 934655 T4535 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={version=2&action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:38661_&wt=javabin&onlyIfLeaderActive=true&checkLive=true&coreNodeName=core_node2&onlyIfLeader=true&state=recovering} status=0 QTime=1003
[junit4] 2> 934876 T4495 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48723_
[junit4] 2> 934878 T4495 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48723_
[junit4] 2> 934893 T4580 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 934894 T4580 oasc.ZkController.publish
[...truncated too long message...]
solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty4/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty4/index;done=false>>]
[junit4] 2> 995779 T4599 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001/tempDir-001/jetty4/index
[junit4] 2> 995779 T4599 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:243)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:346)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:330)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:190)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 995781 T4599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 995781 T4599 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 995781 T4599 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 995782 T4599 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.SyncSliceTest-7733E4D74C9F7CFD-001
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=38), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=38)), sim=DefaultSimilarity, locale=ko_KR, timezone=America/Nome
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=8,threads=1,free=164318568,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [RegexBoostProcessorTest, CoreAdminHandlerTest, DistributedDebugComponentTest, DistribCursorPagingTest, TestReplicationHandler, UpdateParamsTest, BasicFunctionalityTest, TestRemoteStreaming, SchemaVersionSpecificBehaviorTest, ZkControllerTest, AsyncMigrateRouteKeyTest, TestExceedMaxTermLength, TestFunctionQuery, TestGroupingSearch, MigrateRouteKeyTest, JsonLoaderTest, TestCharFilters, CSVRequestHandlerTest, CursorPagingTest, TestLuceneMatchVersion, TestInfoStreamLogging, TestRandomDVFaceting, SliceStateUpdateTest, TestPseudoReturnFields, TestCustomSort, TestSolrIndexConfig, SolrXmlInZkTest, UUIDFieldTest, TestManagedResourceStorage, TermVectorComponentTest, DeleteShardTest, CacheHeaderTest, TestQueryUtils, LoggingHandlerTest, FieldMutatingUpdateProcessorTest, SynonymTokenizerTest, ParsingFieldUpdateProcessorsTest, DistributedQueryComponentCustomSortTest, SOLR749Test, StatelessScriptUpdateProcessorFactoryTest, RemoteQueryErrorTest, MBeansHandlerTest, TestReloadAndDeleteDocs, TestManagedSynonymFilterFactory, ClusterStateUpdateTest, TestNRTOpen, TestNonDefinedSimilarityFactory, CopyFieldTest, TestReversedWildcardFilterFactory, SpatialFilterTest, OverseerRolesTest, SoftAutoCommitTest, CircularListTest, DocValuesTest, BadComponentTest, ShardRoutingTest, RankQueryTest, AutoCommitTest, TestFieldTypeResource, SampleTest, BasicDistributedZkTest, UnloadDistributedZkTest, TestValueSourceCache, ResponseLogComponentTest, TestFaceting, ScriptEngineTest, DateRangeFieldTest, SolrInfoMBeanTest, CollectionsAPIAsyncDistributedZkTest, FileBasedSpellCheckerTest, QueryElevationComponentTest, HdfsUnloadDistributedZkTest, DistributedSuggestComponentTest, TestClassNameShortening, TestRestManager, TestBM25SimilarityFactory, TestSolrXml, TestConfig, TestStandardQParsers, MultiThreadedOCPTest, TestDefaultSimilarityFactory, DistributedSpellCheckComponentTest, MergeStrategyTest, DistributedTermsComponentTest, TestCoreContainer, TestAnalyzeInfixSuggestions, EchoParamsTest, PeerSyncTest, IndexSchemaTest, TestMaxScoreQueryParser, TestSearchPerf, TestExtendedDismaxParser, LeaderElectionTest, TestCollectionAPI, TestImplicitCoreProperties, DirectUpdateHandlerOptimizeTest, SyncSliceTest]
[junit4] Completed on J0 in 69.63s, 1 test, 1 error <<< FAILURES!
[...truncated 630 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1692 tests, 1 error, 36 ignored (15 assumptions)
Total time: 62 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b15) -
Build # 10513 - Still Failing!
Posted by Anshum Gupta <an...@anshumgupta.net>.
Committed a fix.
On Mon, Jun 9, 2014 at 12:56 AM, Policeman Jenkins Server
<je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10513/
> Java: 32bit/jdk1.8.0_20-ea-b15 -server -XX:+UseParallelGC
>
> 1 tests failed.
> FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
>
> Error Message:
> After invoking OVERSEERSTATUS, SplitShard task [2000] was still supposed to be in [running] but isn't.It is [completed]
>
> Stack Trace:
> java.lang.AssertionError: After invoking OVERSEERSTATUS, SplitShard task [2000] was still supposed to be in [running] but isn't.It is [completed]
> at __randomizedtesting.SeedInfo.seed([876CCE9B20D6C54E:68A40835789A572]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.apache.solr.cloud.MultiThreadedOCPTest.testLongAndShortRunningParallelApiCalls(MultiThreadedOCPTest.java:209)
> at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 11041 lines...]
> [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
> [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/init-core-data-001
> [junit4] 2> 350379 T2109 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
> [junit4] 2> 350380 T2109 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
> [junit4] 2> 350394 T2109 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4] 2> 350409 T2109 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4] 1> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 350430 T2110 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4] 2> 350518 T2109 oasc.ZkTestServer.run start zk server on port:51949
> [junit4] 2> 350533 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 350642 T2116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1798141 name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 350642 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 350642 T2109 oascc.SolrZkClient.makePath makePath: /solr
> [junit4] 2> 350653 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 350655 T2118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e90b5a name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 350655 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 350656 T2109 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4] 2> 350658 T2109 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4] 2> 350660 T2109 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4] 2> 350662 T2109 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4] 2> 350680 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4] 2> 350680 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4] 2> 350697 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4] 2> 350698 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4] 2> 350716 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 350717 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 350721 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4] 2> 350722 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4] 2> 350725 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4] 2> 350726 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4] 2> 350729 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4] 2> 350730 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4] 2> 350733 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
> [junit4] 2> 350733 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
> [junit4] 2> 350736 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 350736 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4] 2> 350738 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 350739 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 350743 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4] 2> 350743 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4] 2> 350747 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4] 2> 350747 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4] 2> 351023 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 351027 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60423
> [junit4] 2> 351027 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 351028 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 351028 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002
> [junit4] 2> 351028 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/'
> [junit4] 2> 351039 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/solr.xml
> [junit4] 2> 351047 T2109 oasc.CoreContainer.<init> New CoreContainer 13020690
> [junit4] 2> 351047 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/]
> [junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 351050 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 351050 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 351050 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 351051 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 351051 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
> [junit4] 2> 351051 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 351052 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 351069 T2129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@994f9f name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 351070 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 351080 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 351081 T2131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f4a805 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 351081 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 351084 T2109 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4] 2> 351087 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4] 2> 351110 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
> [junit4] 2> 351120 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
> [junit4] 2> 351125 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
> [junit4] 2> 351130 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4] 2> 351146 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60423_
> [junit4] 2> 351147 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60423_
> [junit4] 2> 351149 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect
> [junit4] 2> 351150 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4] 2> 351165 T2109 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:60423_
> [junit4] 2> 351165 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4] 2> 351168 T2109 oasc.Overseer.start Overseer (id=91901056920322051-127.0.0.1:60423_-n_0000000000) starting
> [junit4] 2> 351171 T2109 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4] 2> 351183 T2133 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4] 2> 351184 T2109 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4] 2> 351192 T2109 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4] 2> 351193 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 351195 T2132 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4] 2> 351198 T2134 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
> [junit4] 2> 351198 T2134 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 351199 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 351199 T2134 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 351200 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 351200 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:60423",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:60423_",
> [junit4] 2> "shard":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 351201 T2132 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
> [junit4] 2> 351201 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 351204 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 351205 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 352201 T2134 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 352202 T2134 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4] 2> 352202 T2134 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 352202 T2134 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
> [junit4] 2> 352203 T2134 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 352203 T2134 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/'
> [junit4] 2> 352204 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/.svn/' to classloader
> [junit4] 2> 352204 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/classes/' to classloader
> [junit4] 2> 352205 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/README' to classloader
> [junit4] 2> 352239 T2134 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
> [junit4] 2> 352253 T2134 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 352268 T2134 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 352274 T2134 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 352374 T2134 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 352381 T2134 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 352382 T2134 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 352408 T2134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 352410 T2134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 352413 T2134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 352415 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 352416 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 352416 T2134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 352420 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 352421 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 352421 T2134 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
> [junit4] 2> 352421 T2134 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 352421 T2134 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/
> [junit4] 2> 352422 T2134 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
> [junit4] 2> 352423 T2134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data
> [junit4] 2> 352424 T2134 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index/
> [junit4] 2> 352424 T2134 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
> [junit4] 2> 352424 T2134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index
> [junit4] 2> 352424 T2134 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
> [junit4] 2> 352426 T2134 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1cb195c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d1ce36),segFN=segments_1,generation=1}
> [junit4] 2> 352426 T2134 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 352431 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 352434 T2134 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 352434 T2134 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 352437 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 352439 T2134 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 352441 T2134 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 352441 T2134 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 352442 T2134 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 352445 T2134 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1cb195c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d1ce36),segFN=segments_1,generation=1}
> [junit4] 2> 352446 T2134 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 352446 T2134 oass.SolrIndexSearcher.<init> Opening Searcher@194d266[collection1] main
> [junit4] 2> 352447 T2134 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
> [junit4] 2> 352447 T2134 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 352447 T2134 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
> [junit4] 2> 352448 T2134 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
> [junit4] 2> 352448 T2134 oasr.RestManager.init Initializing RestManager with initArgs: {}
> [junit4] 2> 352448 T2134 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 352448 T2134 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
> [junit4] 2> 352449 T2134 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 352449 T2134 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
> [junit4] 2> 352449 T2134 oasr.RestManager.init Initializing 0 registered ManagedResources
> [junit4] 2> 352450 T2135 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@194d266[collection1] main{UninvertingDirectoryReader()}
> [junit4] 2> 352451 T2134 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 352453 T2138 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60423 collection:control_collection shard:shard1
> [junit4] 2> 352453 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 352453 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 352455 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 352455 T2138 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4] 2> 352459 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 352460 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 352461 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 352474 T2109 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
> [junit4] 2> 352477 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 352478 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352479 T2138 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 352479 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> 352479 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> ASYNC NEW_CORE C335 name=collection1 org.apache.solr.core.SolrCore@1676b88 url=http://127.0.0.1:60423/collection1 node=127.0.0.1:60423_ C335_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:60423, core=collection1, node_name=127.0.0.1:60423_}
> [junit4] 2> 352479 T2138 C335 P60423 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60423/collection1/
> [junit4] 2> 352480 T2138 C335 P60423 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 352480 T2138 C335 P60423 oasc.SyncStrategy.syncToMe http://127.0.0.1:60423/collection1/ has no replicas
> [junit4] 2> 352481 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60423/collection1/ shard1
> [junit4] 2> 352481 T2138 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4] 2> 352502 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352514 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352517 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352620 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 352620 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 352664 T2138 oasc.ZkController.register We are http://127.0.0.1:60423/collection1/ and leader is http://127.0.0.1:60423/collection1/
> [junit4] 2> 352665 T2138 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60423
> [junit4] 2> 352665 T2138 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 352665 T2138 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
> [junit4] 2> 352665 T2138 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352680 T2138 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 352680 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 352681 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:60423",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:60423_",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 352710 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 352816 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
> [junit4] 2> 352817 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 352818 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47611
> [junit4] 2> 352819 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 352819 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 352819 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003
> [junit4] 2> 352820 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/'
> [junit4] 2> 352821 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 352821 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 352832 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/solr.xml
> [junit4] 2> 352839 T2109 oasc.CoreContainer.<init> New CoreContainer 20568719
> [junit4] 2> 352839 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/]
> [junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 352843 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 352843 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 352844 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 352844 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
> [junit4] 2> 352845 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 352846 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 352861 T2151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1519d4f name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 352862 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 352868 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 352868 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 352868 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 352873 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 353875 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47611_
> [junit4] 2> 353876 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47611_
> [junit4] 2> 353881 T2154 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
> [junit4] 2> 353882 T2154 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 353883 T2154 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 353884 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:47611",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:47611_",
> [junit4] 2> "shard":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
> [junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4] 2> 353886 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 353987 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 353988 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 353987 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 354883 T2154 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 354883 T2154 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 354884 T2154 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 354884 T2154 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 354885 T2154 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 354885 T2154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/'
> [junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/.svn/' to classloader
> [junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/classes/' to classloader
> [junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/README' to classloader
> [junit4] 2> 354910 T2154 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
> [junit4] 2> 354917 T2154 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 354918 T2154 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 354922 T2154 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 354995 T2154 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 354999 T2154 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 355000 T2154 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 355007 T2154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 355009 T2154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 355010 T2154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 355012 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 355012 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 355012 T2154 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
> [junit4] 2> 355013 T2154 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 355013 T2154 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/
> [junit4] 2> 355013 T2154 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
> [junit4] 2> 355014 T2154 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data
> [junit4] 2> 355014 T2154 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index/
> [junit4] 2> 355014 T2154 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
> [junit4] 2> 355014 T2154 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index
> [junit4] 2> 355014 T2154 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
> [junit4] 2> 355015 T2154 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1f42922 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1440fdb),segFN=segments_1,generation=1}
> [junit4] 2> 355015 T2154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 355019 T2154 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 355021 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 355022 T2154 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 355023 T2154 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 355023 T2154 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 355024 T2154 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 355024 T2154 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1f42922 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1440fdb),segFN=segments_1,generation=1}
> [junit4] 2> 355024 T2154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 355024 T2154 oass.SolrIndexSearcher.<init> Opening Searcher@17000a2[collection1] main
> [junit4] 2> 355025 T2154 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 355025 T2154 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 355025 T2154 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
> [junit4] 2> 355026 T2154 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
> [junit4] 2> 355026 T2154 oasr.RestManager.init Initializing RestManager with initArgs: {}
> [junit4] 2> 355026 T2154 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 355027 T2154 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
> [junit4] 2> 355027 T2154 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 355027 T2154 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
> [junit4] 2> 355027 T2154 oasr.RestManager.init Initializing 0 registered ManagedResources
> [junit4] 2> 355036 T2155 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17000a2[collection1] main{UninvertingDirectoryReader()}
> [junit4] 2> 355037 T2154 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 355038 T2158 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47611 collection:collection1 shard:shard2
> [junit4] 2> 355038 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 355039 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 355039 T2158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4] 2> 355043 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355050 T2158 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 355050 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> 355050 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> ASYNC NEW_CORE C336 name=collection1 org.apache.solr.core.SolrCore@1e1ec0a url=http://127.0.0.1:47611/collection1 node=127.0.0.1:47611_ C336_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47611, core=collection1, node_name=127.0.0.1:47611_}
> [junit4] 2> 355050 T2158 C336 P47611 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47611/collection1/
> [junit4] 2> 355051 T2158 C336 P47611 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 355053 T2158 C336 P47611 oasc.SyncStrategy.syncToMe http://127.0.0.1:47611/collection1/ has no replicas
> [junit4] 2> 355054 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47611/collection1/ shard2
> [junit4] 2> 355054 T2158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4] 2> 355054 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355057 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355059 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355163 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355163 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355163 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355209 T2158 oasc.ZkController.register We are http://127.0.0.1:47611/collection1/ and leader is http://127.0.0.1:47611/collection1/
> [junit4] 2> 355209 T2158 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47611
> [junit4] 2> 355210 T2158 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 355210 T2158 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
> [junit4] 2> 355210 T2158 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355213 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 355213 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 355214 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:47611",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:47611_",
> [junit4] 2> "shard":"shard2",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 355215 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 355276 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
> [junit4] 2> 355277 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 355279 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42158
> [junit4] 2> 355280 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 355280 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 355280 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004
> [junit4] 2> 355281 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/'
> [junit4] 2> 355291 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/solr.xml
> [junit4] 2> 355297 T2109 oasc.CoreContainer.<init> New CoreContainer 216655
> [junit4] 2> 355297 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/]
> [junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 355301 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 355301 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 355301 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 355302 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
> [junit4] 2> 355302 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 355303 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 355304 T2169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@112b6c3 name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 355304 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 355306 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 355306 T2171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ff266 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 355306 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 355310 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 355316 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355316 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355316 T2171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 355316 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 356312 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42158_
> [junit4] 2> 356314 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42158_
> [junit4] 2> 356319 T2172 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
> [junit4] 2> 356319 T2172 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 356327 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 356327 T2172 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 356327 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 356328 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 356328 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:42158",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:42158_",
> [junit4] 2> "shard":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 356331 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 356433 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 356433 T2171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 356433 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 356433 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 357327 T2172 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 357328 T2172 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 357328 T2172 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 357329 T2172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 357329 T2172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 357329 T2172 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/'
> [junit4] 2> 357330 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/.svn/' to classloader
> [junit4] 2> 357330 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/classes/' to classloader
> [junit4] 2> 357331 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/README' to classloader
> [junit4] 2> 357354 T2172 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
> [junit4] 2> 357365 T2172 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 357366 T2172 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 357372 T2172 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 357481 T2172 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 357486 T2172 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 357487 T2172 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 357499 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 357501 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 357503 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 357503 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 357504 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 357504 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 357506 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4] 2> 357506 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4] 2> 357506 T2172 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
> [junit4] 2> 357506 T2172 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 357507 T2172 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/
> [junit4] 2> 357507 T2172 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
> [junit4] 2> 357508 T2172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data
> [junit4] 2> 357508 T2172 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index/
> [junit4] 2> 357508 T2172 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
> [junit4] 2> 357508 T2172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index
> [junit4] 2> 357509 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
> [junit4] 2> 357510 T2172 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1593c84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@264ec9),segFN=segments_1,generation=1}
> [junit4] 2> 357510 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 357513 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 357513 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 357514 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 357514 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 357515 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 357515 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 357516 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 357516 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 357516 T2172 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 357519 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 357519 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 357521 T2172 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 357523 T2172 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 357523 T2172 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 357524 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 357524 T2172 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1593c84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@264ec9),segFN=segments_1,generation=1}
> [junit4] 2> 357524 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 357525 T2172 oass.SolrIndexSearcher.<init> Opening Searcher@653284[collection1] main
> [junit4] 2> 357525 T2172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 357526 T2172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 357526 T2172 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
> [junit4] 2> 357527 T2172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
> [junit4] 2> 357527 T2172 oasr.RestManager.init Initializing RestManager with initArgs: {}
> [junit4] 2> 357527 T2172 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 357528 T2172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
> [junit4] 2> 357528 T2172 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 357528 T2172 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
> [junit4] 2> 357528 T2172 oasr.RestManager.init Initializing 0 registered ManagedResources
> [junit4] 2> 357531 T2173 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@653284[collection1] main{UninvertingDirectoryReader()}
> [junit4] 2> 357532 T2172 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 357533 T2176 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42158 collection:collection1 shard:shard1
> [junit4] 2> 357533 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 357533 T2176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 357533 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 357536 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357538 T2176 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 357538 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C337 name=collection1 org.apache.solr.core.SolrCore@e077c4 url=http://127.0.0.1:42158/collection1 node=127.0.0.1:42158_ C337_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:42158, core=collection1, node_name=127.0.0.1:42158_}
> [junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42158/collection1/
> [junit4] 2> 357539 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.syncToMe http://127.0.0.1:42158/collection1/ has no replicas
> [junit4] 2> 357539 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42158/collection1/ shard1
> [junit4] 2> 357540 T2176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4] 2> 357540 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357542 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357545 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 357803 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
> [junit4] 2> 357804 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 357806 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41109
> [junit4] 2> 357807 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 357808 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 357808 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005
> [junit4] 2> 357808 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/'
> [junit4] 2> 357818 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/solr.xml
> [junit4] 2> 357824 T2109 oasc.CoreContainer.<init> New CoreContainer 24516345
> [junit4] 2> 357825 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/]
> [junit4] 2> 357825 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 357825 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 357828 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 357828 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 357828 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 357829 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
> [junit4] 2> 357829 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 357830 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 363103 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380004, likely client has closed socket
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2>
> [junit4] 2> 363203 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
> [junit4] 2> 363204 T2140 oascc.ConnectionManager.process zkClient has disconnected
> [junit4] 2> 364753 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 369774 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380006, likely client has closed socket
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2>
> [junit4] 2> 369874 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
> [junit4] 2> 369875 T2153 oascc.ConnectionManager.process zkClient has disconnected
> [junit4] 2> 370883 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380008, likely client has closed socket
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2>
> [junit4] 2> 370984 T2171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ff266 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
> [junit4] 2> 370984 T2171 oascc.ConnectionManager.process zkClient has disconnected
> [junit4] 2> 370984 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380003, likely client has closed socket
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2>
> [junit4] 2> 371050 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371084 T2131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f4a805 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
> [junit4] 2> 371084 T2131 oascc.ConnectionManager.process zkClient has disconnected
> [junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
> [junit4] 2> 371422 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380004, likely client has closed socket
> [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at org.apache
>
> [...truncated too long message...]
>
> erException.java:51)
> [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> [junit4] 2>
> [junit4] 2> ASYNC NEW_CORE C441 name=collection1 org.apache.solr.core.SolrCore@1ed1f7d url=http://127.0.0.1:33972/collection1 node=127.0.0.1:33972_ C441_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:33972, core=collection1, node_name=127.0.0.1:33972_}
> [junit4] 2> 602470 T2209 C441 P33972 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
> [junit4] 2> 602471 T2209 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
> [junit4] 2> 602471 T2209 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1ed1f7d
> [junit4] 2> 602475 T2209 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=371,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=371,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
> [junit4] 2> 602475 T2209 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4] 2> 602475 T2209 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4] 2> 602475 T2209 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4] 2> 602475 T2209 C441 P33972 oasu.DirectUpdateHandler2.closeWriter Committing on IndexWriter close.
> [junit4] 2> 602476 T2209 C441 P33972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e57ed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@e9b4cd),segFN=segments_1,generation=1}
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e57ed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@e9b4cd),segFN=segments_2,generation=2}
> [junit4] 2> 602476 T2209 C441 P33972 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4] 2> 602477 T2209 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4] 2> 602477 T2209 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
> [junit4] 2> 602477 T2209 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data;done=false>>]
> [junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data
> [junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index;done=false>>]
> [junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index
> [junit4] 2> 602478 T2209 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> [junit4] 2>
> [junit4] 2> 602478 T2209 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001
> [junit4] 2> NOTE: test params are: codec=Lucene49: {_version_=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), multiDefault=FSTOrd41, intDefault=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), id=PostingsFormat(name=NestedPulsing), range_facet_l=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), timestamp=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41)}, docValues:{timestamp=DocValuesFormat(name=Lucene49)}, sim=DefaultSimilarity, locale=nl, timezone=Antarctica/Troll
> [junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_20-ea (32-bit)/cpus=8,threads=1,free=144613768,total=341311488
> [junit4] 2> NOTE: All tests run in this JVM: [DocumentAnalysisRequestHandlerTest, TestDocBasedVersionConstraints, TestBinaryResponseWriter, OverseerRolesTest, NoFacetTest, TestFreeTextSuggestions, SolrPluginUtilsTest, HighlighterTest, TestCollationField, SolrRequestParserTest, OutputWriterTest, TestQuerySenderNoQuery, TestNoOpRegenerator, TestWordDelimiterFilterFactory, TestSolr4Spatial, NotRequiredUniqueKeyTest, TestFuzzyAnalyzedSuggestions, ShardRoutingCustomTest, TestReplicationHandlerBackup, SSLMigrationTest, DeleteReplicaTest, TestSolrJ, TestRecoveryHdfs, IndexSchemaRuntimeFieldTest, SpellCheckCollatorTest, DefaultValueUpdateProcessorTest, SimplePostToolTest, TestBinaryField, StatsComponentTest, CurrencyFieldOpenExchangeTest, TestReload, UniqFieldsUpdateProcessorFactoryTest, TestStressReorder, NoCacheHeaderTest, FileUtilsTest, JSONWriterTest, TestCoreDiscovery, HdfsDirectoryTest, AtomicUpdatesTest, TestDefaultSearchFieldResource, SolrXmlInZkTest, TestMultiCoreConfBootstrap, TestDistribDocBasedVersion, TestStressLucene, TestSolrCoreProperties, AbstractAnalyticsStatsTest, TestChildDocTransformer, TestDynamicFieldResource, FieldAnalysisRequestHandlerTest, DocValuesMultiTest, TestFieldTypeCollectionResource, ConvertedLegacyTest, HdfsBasicDistributedZk2Test, XsltUpdateRequestHandlerTest, TestFastOutputStream, TestLRUCache, TriLevelCompositeIdRoutingTest, DistributedSuggestComponentTest, MultiThreadedOCPTest]
> [junit4] Completed on J1 in 252.74s, 1 test, 1 failure <<< FAILURES!
>
> [...truncated 858 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 failure, 36 ignored (15 assumptions)
>
> Total time: 81 minutes 12 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 32bit/jdk1.8.0_20-ea-b15 -server -XX:+UseParallelGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
--
Anshum Gupta
http://www.anshumgupta.net
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b15) -
Build # 10513 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10513/
Java: 32bit/jdk1.8.0_20-ea-b15 -server -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
After invoking OVERSEERSTATUS, SplitShard task [2000] was still supposed to be in [running] but isn't.It is [completed]
Stack Trace:
java.lang.AssertionError: After invoking OVERSEERSTATUS, SplitShard task [2000] was still supposed to be in [running] but isn't.It is [completed]
at __randomizedtesting.SeedInfo.seed([876CCE9B20D6C54E:68A40835789A572]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testLongAndShortRunningParallelApiCalls(MultiThreadedOCPTest.java:209)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11041 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/init-core-data-001
[junit4] 2> 350379 T2109 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 350380 T2109 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 350394 T2109 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 350409 T2109 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 350430 T2110 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 350518 T2109 oasc.ZkTestServer.run start zk server on port:51949
[junit4] 2> 350533 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 350642 T2116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1798141 name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 350642 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 350642 T2109 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 350653 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 350655 T2118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e90b5a name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 350655 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 350656 T2109 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 350658 T2109 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 350660 T2109 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 350662 T2109 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 350680 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 350680 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 350697 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 350698 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 350716 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 350717 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 350721 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 350722 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 350725 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 350726 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 350729 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 350730 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 350733 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 350733 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 350736 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 350736 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 350738 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 350739 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 350743 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 350743 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 350747 T2109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 350747 T2109 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 351023 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 351027 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60423
[junit4] 2> 351027 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 351028 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 351028 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002
[junit4] 2> 351028 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/'
[junit4] 2> 351039 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/solr.xml
[junit4] 2> 351047 T2109 oasc.CoreContainer.<init> New CoreContainer 13020690
[junit4] 2> 351047 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/]
[junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 351048 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 351049 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 351050 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 351050 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 351050 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 351051 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 351051 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
[junit4] 2> 351051 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 351052 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 351069 T2129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@994f9f name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 351070 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 351080 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 351081 T2131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f4a805 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 351081 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 351084 T2109 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 351087 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 351110 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 351120 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 351125 T2109 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 351130 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 351146 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60423_
[junit4] 2> 351147 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60423_
[junit4] 2> 351149 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 351150 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 351165 T2109 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:60423_
[junit4] 2> 351165 T2109 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 351168 T2109 oasc.Overseer.start Overseer (id=91901056920322051-127.0.0.1:60423_-n_0000000000) starting
[junit4] 2> 351171 T2109 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 351183 T2133 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 351184 T2109 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 351192 T2109 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 351193 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 351195 T2132 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 351198 T2134 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 351198 T2134 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 351199 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 351199 T2134 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 351200 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 351200 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60423",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60423_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 351201 T2132 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 351201 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 351204 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 351205 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 352201 T2134 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 352202 T2134 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 352202 T2134 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 352202 T2134 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 352203 T2134 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 352203 T2134 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/'
[junit4] 2> 352204 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 352204 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 352205 T2134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 352239 T2134 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 352253 T2134 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 352268 T2134 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 352274 T2134 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 352374 T2134 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 352381 T2134 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 352382 T2134 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 352408 T2134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 352410 T2134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 352413 T2134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 352415 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 352416 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 352416 T2134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 352420 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 352421 T2134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 352421 T2134 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 352421 T2134 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 352421 T2134 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/
[junit4] 2> 352422 T2134 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
[junit4] 2> 352423 T2134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data
[junit4] 2> 352424 T2134 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index/
[junit4] 2> 352424 T2134 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 352424 T2134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-001/control/data/index
[junit4] 2> 352424 T2134 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
[junit4] 2> 352426 T2134 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1cb195c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d1ce36),segFN=segments_1,generation=1}
[junit4] 2> 352426 T2134 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 352431 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 352432 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 352433 T2134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 352434 T2134 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 352434 T2134 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 352435 T2134 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 352436 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 352437 T2134 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 352439 T2134 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 352441 T2134 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 352441 T2134 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 352442 T2134 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 352445 T2134 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1cb195c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d1ce36),segFN=segments_1,generation=1}
[junit4] 2> 352446 T2134 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 352446 T2134 oass.SolrIndexSearcher.<init> Opening Searcher@194d266[collection1] main
[junit4] 2> 352447 T2134 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 352447 T2134 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 352447 T2134 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 352448 T2134 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 352448 T2134 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 352448 T2134 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 352448 T2134 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 352449 T2134 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 352449 T2134 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 352449 T2134 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 352450 T2135 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@194d266[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 352451 T2134 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 352453 T2138 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60423 collection:control_collection shard:shard1
[junit4] 2> 352453 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 352453 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 352455 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 352455 T2138 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 352459 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 352460 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 352461 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 352474 T2109 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 352477 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 352478 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352479 T2138 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 352479 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 352479 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C335 name=collection1 org.apache.solr.core.SolrCore@1676b88 url=http://127.0.0.1:60423/collection1 node=127.0.0.1:60423_ C335_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:60423, core=collection1, node_name=127.0.0.1:60423_}
[junit4] 2> 352479 T2138 C335 P60423 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60423/collection1/
[junit4] 2> 352480 T2138 C335 P60423 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 352480 T2138 C335 P60423 oasc.SyncStrategy.syncToMe http://127.0.0.1:60423/collection1/ has no replicas
[junit4] 2> 352481 T2138 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60423/collection1/ shard1
[junit4] 2> 352481 T2138 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 352502 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352514 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352517 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352620 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 352620 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 352664 T2138 oasc.ZkController.register We are http://127.0.0.1:60423/collection1/ and leader is http://127.0.0.1:60423/collection1/
[junit4] 2> 352665 T2138 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60423
[junit4] 2> 352665 T2138 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 352665 T2138 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 352665 T2138 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352679 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352680 T2138 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 352680 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 352681 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:60423",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60423_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 352710 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 352816 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 352817 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 352818 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47611
[junit4] 2> 352819 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 352819 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 352819 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003
[junit4] 2> 352820 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/'
[junit4] 2> 352821 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 352821 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 352832 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/solr.xml
[junit4] 2> 352839 T2109 oasc.CoreContainer.<init> New CoreContainer 20568719
[junit4] 2> 352839 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/]
[junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 352840 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 352841 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 352842 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 352843 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 352843 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 352844 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 352844 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
[junit4] 2> 352845 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 352846 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 352861 T2151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1519d4f name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 352862 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 352868 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 352868 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 352868 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 352873 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 353875 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47611_
[junit4] 2> 353876 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47611_
[junit4] 2> 353881 T2154 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 353882 T2154 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 353883 T2154 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 353883 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 353884 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47611",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47611_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 353884 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 353886 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 353987 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 353988 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 353987 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 354883 T2154 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 354883 T2154 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 354884 T2154 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 354884 T2154 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 354885 T2154 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 354885 T2154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/'
[junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 354886 T2154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 354910 T2154 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 354917 T2154 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 354918 T2154 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 354922 T2154 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 354995 T2154 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 354999 T2154 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 355000 T2154 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 355007 T2154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 355009 T2154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 355010 T2154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 355011 T2154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 355012 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 355012 T2154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 355012 T2154 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 355013 T2154 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 355013 T2154 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/
[junit4] 2> 355013 T2154 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
[junit4] 2> 355014 T2154 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data
[junit4] 2> 355014 T2154 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index/
[junit4] 2> 355014 T2154 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 355014 T2154 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-003/collection1/data/index
[junit4] 2> 355014 T2154 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
[junit4] 2> 355015 T2154 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1f42922 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1440fdb),segFN=segments_1,generation=1}
[junit4] 2> 355015 T2154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 355017 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 355018 T2154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 355019 T2154 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 355019 T2154 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 355020 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 355021 T2154 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 355022 T2154 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 355023 T2154 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 355023 T2154 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 355024 T2154 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 355024 T2154 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1f42922 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1440fdb),segFN=segments_1,generation=1}
[junit4] 2> 355024 T2154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 355024 T2154 oass.SolrIndexSearcher.<init> Opening Searcher@17000a2[collection1] main
[junit4] 2> 355025 T2154 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 355025 T2154 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 355025 T2154 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 355026 T2154 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 355026 T2154 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 355026 T2154 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 355027 T2154 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 355027 T2154 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 355027 T2154 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 355027 T2154 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 355036 T2155 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17000a2[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 355037 T2154 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 355038 T2158 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47611 collection:collection1 shard:shard2
[junit4] 2> 355038 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 355039 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 355039 T2158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 355043 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355046 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355050 T2158 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 355050 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 355050 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C336 name=collection1 org.apache.solr.core.SolrCore@1e1ec0a url=http://127.0.0.1:47611/collection1 node=127.0.0.1:47611_ C336_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47611, core=collection1, node_name=127.0.0.1:47611_}
[junit4] 2> 355050 T2158 C336 P47611 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47611/collection1/
[junit4] 2> 355051 T2158 C336 P47611 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 355053 T2158 C336 P47611 oasc.SyncStrategy.syncToMe http://127.0.0.1:47611/collection1/ has no replicas
[junit4] 2> 355054 T2158 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47611/collection1/ shard2
[junit4] 2> 355054 T2158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 355054 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355057 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355059 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355163 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355163 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355163 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355209 T2158 oasc.ZkController.register We are http://127.0.0.1:47611/collection1/ and leader is http://127.0.0.1:47611/collection1/
[junit4] 2> 355209 T2158 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47611
[junit4] 2> 355210 T2158 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 355210 T2158 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 355210 T2158 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355212 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355213 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 355213 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 355214 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47611",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47611_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 355215 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 355276 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 355277 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 355279 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42158
[junit4] 2> 355280 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 355280 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 355280 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004
[junit4] 2> 355281 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/'
[junit4] 2> 355291 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/solr.xml
[junit4] 2> 355297 T2109 oasc.CoreContainer.<init> New CoreContainer 216655
[junit4] 2> 355297 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/]
[junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 355298 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 355299 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 355300 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 355301 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 355301 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 355301 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 355302 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
[junit4] 2> 355302 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 355303 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 355304 T2169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@112b6c3 name:ZooKeeperConnection Watcher:127.0.0.1:51949 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 355304 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 355306 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 355306 T2171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ff266 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 355306 T2109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 355310 T2109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 355316 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355316 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355316 T2171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 355316 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 356312 T2109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42158_
[junit4] 2> 356314 T2109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42158_
[junit4] 2> 356319 T2172 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 356319 T2172 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 356327 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 356327 T2172 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 356327 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 356328 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 356328 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42158",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42158_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 356329 T2132 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 356331 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 356433 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 356433 T2171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 356433 T2140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 356433 T2153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 357327 T2172 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 357328 T2172 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 357328 T2172 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 357329 T2172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 357329 T2172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 357329 T2172 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/'
[junit4] 2> 357330 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 357330 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 357331 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 357354 T2172 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 357365 T2172 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 357366 T2172 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 357372 T2172 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 357481 T2172 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 357486 T2172 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 357487 T2172 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 357499 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 357501 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 357503 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 357503 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 357504 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 357504 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 357506 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 357506 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 357506 T2172 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 357506 T2172 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 357507 T2172 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/
[junit4] 2> 357507 T2172 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@eebee1
[junit4] 2> 357508 T2172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data
[junit4] 2> 357508 T2172 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index/
[junit4] 2> 357508 T2172 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 357508 T2172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-004/collection1/data/index
[junit4] 2> 357509 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7919802637305674]
[junit4] 2> 357510 T2172 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1593c84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@264ec9),segFN=segments_1,generation=1}
[junit4] 2> 357510 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 357513 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 357513 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 357514 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 357514 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 357515 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 357515 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 357516 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 357516 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 357516 T2172 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 357517 T2172 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 357518 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 357519 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 357519 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 357521 T2172 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 357523 T2172 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 357523 T2172 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 357524 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=28.703125, floorSegmentMB=0.94140625, forceMergeDeletesPctAllowed=15.153936044953278, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 357524 T2172 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1593c84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@264ec9),segFN=segments_1,generation=1}
[junit4] 2> 357524 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 357525 T2172 oass.SolrIndexSearcher.<init> Opening Searcher@653284[collection1] main
[junit4] 2> 357525 T2172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 357526 T2172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 357526 T2172 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 357527 T2172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 357527 T2172 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 357527 T2172 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 357528 T2172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 357528 T2172 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 357528 T2172 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 357528 T2172 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 357531 T2173 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@653284[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 357532 T2172 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 357533 T2176 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42158 collection:collection1 shard:shard1
[junit4] 2> 357533 T2109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 357533 T2176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 357533 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 357536 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357538 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357538 T2176 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 357538 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C337 name=collection1 org.apache.solr.core.SolrCore@e077c4 url=http://127.0.0.1:42158/collection1 node=127.0.0.1:42158_ C337_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:42158, core=collection1, node_name=127.0.0.1:42158_}
[junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42158/collection1/
[junit4] 2> 357539 T2132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 357539 T2176 C337 P42158 oasc.SyncStrategy.syncToMe http://127.0.0.1:42158/collection1/ has no replicas
[junit4] 2> 357539 T2176 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42158/collection1/ shard1
[junit4] 2> 357540 T2176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 357540 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357542 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357545 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 357803 T2109 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 357804 T2109 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 357806 T2109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41109
[junit4] 2> 357807 T2109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 357808 T2109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 357808 T2109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005
[junit4] 2> 357808 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/'
[junit4] 2> 357818 T2109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/solr.xml
[junit4] 2> 357824 T2109 oasc.CoreContainer.<init> New CoreContainer 24516345
[junit4] 2> 357825 T2109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-005/]
[junit4] 2> 357825 T2109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 357825 T2109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 357826 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 357827 T2109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 357828 T2109 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 357828 T2109 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 357828 T2109 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 357829 T2109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51949/solr
[junit4] 2> 357829 T2109 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 357830 T2109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 363103 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 363203 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 363204 T2140 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 364753 T2140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0d759 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 369774 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 369874 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 369875 T2153 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 370883 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 370984 T2171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ff266 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 370984 T2171 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 370984 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 371050 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7602 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371084 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371084 T2131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f4a805 name:ZooKeeperConnection Watcher:127.0.0.1:51949/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 371084 T2131 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371085 T2131 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 371422 T2111 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f863c380004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache
[...truncated too long message...]
erException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> ASYNC NEW_CORE C441 name=collection1 org.apache.solr.core.SolrCore@1ed1f7d url=http://127.0.0.1:33972/collection1 node=127.0.0.1:33972_ C441_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:33972, core=collection1, node_name=127.0.0.1:33972_}
[junit4] 2> 602470 T2209 C441 P33972 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 602471 T2209 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 602471 T2209 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1ed1f7d
[junit4] 2> 602475 T2209 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=371,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=371,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 602475 T2209 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 602475 T2209 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 602475 T2209 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 602475 T2209 C441 P33972 oasu.DirectUpdateHandler2.closeWriter Committing on IndexWriter close.
[junit4] 2> 602476 T2209 C441 P33972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e57ed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@e9b4cd),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e57ed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@e9b4cd),segFN=segments_2,generation=2}
[junit4] 2> 602476 T2209 C441 P33972 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 602477 T2209 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 602477 T2209 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 602477 T2209 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data;done=false>>]
[junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data
[junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index;done=false>>]
[junit4] 2> 602478 T2209 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001/tempDir-006/collection1/data/index
[junit4] 2> 602478 T2209 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 602478 T2209 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-876CCE9B20D6C54E-001
[junit4] 2> NOTE: test params are: codec=Lucene49: {_version_=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), multiDefault=FSTOrd41, intDefault=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), id=PostingsFormat(name=NestedPulsing), range_facet_l=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41), timestamp=Pulsing41(freqCutoff=10 minBlockSize=9 maxBlockSize=41)}, docValues:{timestamp=DocValuesFormat(name=Lucene49)}, sim=DefaultSimilarity, locale=nl, timezone=Antarctica/Troll
[junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_20-ea (32-bit)/cpus=8,threads=1,free=144613768,total=341311488
[junit4] 2> NOTE: All tests run in this JVM: [DocumentAnalysisRequestHandlerTest, TestDocBasedVersionConstraints, TestBinaryResponseWriter, OverseerRolesTest, NoFacetTest, TestFreeTextSuggestions, SolrPluginUtilsTest, HighlighterTest, TestCollationField, SolrRequestParserTest, OutputWriterTest, TestQuerySenderNoQuery, TestNoOpRegenerator, TestWordDelimiterFilterFactory, TestSolr4Spatial, NotRequiredUniqueKeyTest, TestFuzzyAnalyzedSuggestions, ShardRoutingCustomTest, TestReplicationHandlerBackup, SSLMigrationTest, DeleteReplicaTest, TestSolrJ, TestRecoveryHdfs, IndexSchemaRuntimeFieldTest, SpellCheckCollatorTest, DefaultValueUpdateProcessorTest, SimplePostToolTest, TestBinaryField, StatsComponentTest, CurrencyFieldOpenExchangeTest, TestReload, UniqFieldsUpdateProcessorFactoryTest, TestStressReorder, NoCacheHeaderTest, FileUtilsTest, JSONWriterTest, TestCoreDiscovery, HdfsDirectoryTest, AtomicUpdatesTest, TestDefaultSearchFieldResource, SolrXmlInZkTest, TestMultiCoreConfBootstrap, TestDistribDocBasedVersion, TestStressLucene, TestSolrCoreProperties, AbstractAnalyticsStatsTest, TestChildDocTransformer, TestDynamicFieldResource, FieldAnalysisRequestHandlerTest, DocValuesMultiTest, TestFieldTypeCollectionResource, ConvertedLegacyTest, HdfsBasicDistributedZk2Test, XsltUpdateRequestHandlerTest, TestFastOutputStream, TestLRUCache, TriLevelCompositeIdRoutingTest, DistributedSuggestComponentTest, MultiThreadedOCPTest]
[junit4] Completed on J1 in 252.74s, 1 test, 1 failure <<< FAILURES!
[...truncated 858 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 failure, 36 ignored (15 assumptions)
Total time: 81 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_20-ea-b15 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60) - Build #
10512 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10512/
Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
overseerstatus the collection error [Watcher fired on path: null state: SyncConnected type None]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: overseerstatus the collection error [Watcher fired on path: null state: SyncConnected type None]
at __randomizedtesting.SeedInfo.seed([DD151805B732301F:5CF3961DC06D5023]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.invokeCollectionApi(AbstractFullDistribZkTestBase.java:1738)
at org.apache.solr.cloud.MultiThreadedOCPTest.testLongAndShortRunningParallelApiCalls(MultiThreadedOCPTest.java:195)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11594 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/init-core-data-001
[junit4] 2> 1368574 T5636 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 1368575 T5636 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /z/
[junit4] 2> 1368578 T5636 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1368578 T5636 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1368579 T5637 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1368679 T5636 oasc.ZkTestServer.run start zk server on port:45432
[junit4] 2> 1368680 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1393181 T5640 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 24499ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 1393184 T5638 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f01a6e20000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1393185 T5638 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1467f01a6e20001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1393186 T5643 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60f93e86 name:ZooKeeperConnection Watcher:127.0.0.1:45432 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1393187 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1393187 T5636 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1393191 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1393193 T5645 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@294bf6ca name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1393193 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1393194 T5636 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1393199 T5636 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1393202 T5636 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1393205 T5636 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1393208 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1393208 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1393211 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1393212 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1393214 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1393214 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1393216 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1393216 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1393219 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1393219 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1393223 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1393223 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1393227 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1393227 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1393231 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1393232 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1393236 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1393236 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1393240 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1393241 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1393244 T5636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1393245 T5636 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1393250 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1393252 T5647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fc6e55c name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1393252 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1393469 T5636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1393472 T5636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1393476 T5636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35447
[junit4] 2> 1393478 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1393479 T5636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1393479 T5636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002
[junit4] 2> 1393480 T5636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/'
[junit4] 2> 1393495 T5636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/solr.xml
[junit4] 2> 1393516 T5636 oasc.CoreContainer.<init> New CoreContainer 506924388
[junit4] 2> 1393516 T5636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/]
[junit4] 2> 1393517 T5636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1393517 T5636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1393517 T5636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1393517 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1393518 T5636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1393518 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1393518 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1393518 T5636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1393518 T5636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1393520 T5636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1393520 T5636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1393521 T5636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1393521 T5636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45432/solr
[junit4] 2> 1393521 T5636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1393522 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1393523 T5658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62c44449 name:ZooKeeperConnection Watcher:127.0.0.1:45432 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1393524 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1393526 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1393529 T5660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@437bea27 name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1393529 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1393532 T5636 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1393537 T5636 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1393557 T5636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1393561 T5636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1393571 T5636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1393576 T5636 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1393578 T5636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35447_z
[junit4] 2> 1393579 T5636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35447_z
[junit4] 2> 1393582 T5636 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1393585 T5636 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1393589 T5636 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:35447_z
[junit4] 2> 1393590 T5636 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1393593 T5636 oasc.Overseer.start Overseer (id=91900487479197702-127.0.0.1:35447_z-n_0000000000) starting
[junit4] 2> 1393596 T5636 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1393611 T5662 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1393612 T5636 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1393615 T5636 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1393618 T5636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1393621 T5661 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1393626 T5663 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1393627 T5663 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1393628 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1393628 T5663 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1393630 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1393631 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35447/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35447_z",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1393632 T5661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1393632 T5661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1393635 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1393637 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1394629 T5663 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1394629 T5663 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1394630 T5663 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1394631 T5663 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1394632 T5663 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1394632 T5663 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/collection1/'
[junit4] 2> 1394633 T5663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 1394634 T5663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 1394634 T5663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 1394666 T5663 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1394686 T5663 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1394688 T5663 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1394693 T5663 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1394860 T5663 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1394864 T5663 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1394865 T5663 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1394879 T5663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1394884 T5663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1394888 T5663 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1394889 T5663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1394890 T5663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1394890 T5663 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1394891 T5663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1394892 T5663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1394892 T5663 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1394892 T5663 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1394893 T5663 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-001/control/data/
[junit4] 2> 1394893 T5663 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3abea411
[junit4] 2> 1394894 T5663 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-001/control/data
[junit4] 2> 1394894 T5663 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-001/control/data/index/
[junit4] 2> 1394895 T5663 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1394895 T5663 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-001/control/data/index
[junit4] 2> 1394896 T5663 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=740508858, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1394897 T5663 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2beec6f5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65686198),segFN=segments_1,generation=1}
[junit4] 2> 1394897 T5663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1394902 T5663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1394902 T5663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1394903 T5663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1394903 T5663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1394904 T5663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1394904 T5663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1394905 T5663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1394905 T5663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1394905 T5663 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1394906 T5663 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1394906 T5663 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1394907 T5663 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1394907 T5663 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1394908 T5663 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1394908 T5663 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1394909 T5663 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1394909 T5663 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1394913 T5663 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1394917 T5663 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1394917 T5663 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1394918 T5663 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=24.5615234375, floorSegmentMB=1.0751953125, forceMergeDeletesPctAllowed=17.926191470419468, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3957502255189558
[junit4] 2> 1394918 T5663 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2beec6f5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65686198),segFN=segments_1,generation=1}
[junit4] 2> 1394919 T5663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1394919 T5663 oass.SolrIndexSearcher.<init> Opening Searcher@357a5810[collection1] main
[junit4] 2> 1394920 T5663 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1394921 T5663 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1394921 T5663 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1394922 T5663 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1394922 T5663 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1394923 T5663 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1394924 T5663 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1394924 T5663 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1394924 T5663 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 1394925 T5663 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1394927 T5664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@357a5810[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1394929 T5663 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1394930 T5667 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35447/z collection:control_collection shard:shard1
[junit4] 2> 1394930 T5636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1394931 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1394932 T5667 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1394936 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1394938 T5669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4dfdbd84 name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1394938 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1394939 T5636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1394940 T5667 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1394943 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1394944 T5667 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1394944 T5667 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3760 name=collection1 org.apache.solr.core.SolrCore@68a77d28 url=https://127.0.0.1:35447/z/collection1 node=127.0.0.1:35447_z C3760_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:35447/z, core=collection1, node_name=127.0.0.1:35447_z}
[junit4] 2> 1394945 T5667 C3760 P35447 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35447/z/collection1/
[junit4] 2> 1394944 T5636 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1394945 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1394945 T5667 C3760 P35447 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1394946 T5667 C3760 P35447 oasc.SyncStrategy.syncToMe https://127.0.0.1:35447/z/collection1/ has no replicas
[junit4] 2> 1394946 T5667 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35447/z/collection1/ shard1
[junit4] 2> 1394947 T5667 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1394951 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1394957 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1394962 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1395064 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1395064 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1395109 T5667 oasc.ZkController.register We are https://127.0.0.1:35447/z/collection1/ and leader is https://127.0.0.1:35447/z/collection1/
[junit4] 2> 1395109 T5667 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35447/z
[junit4] 2> 1395109 T5667 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1395109 T5667 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1395109 T5667 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1395111 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1395111 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1395111 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1395111 T5667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1395112 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1395113 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35447/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35447_z",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1395115 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1395153 T5636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1395153 T5636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1395156 T5636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1395157 T5636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:49373
[junit4] 2> 1395160 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1395161 T5636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1395161 T5636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003
[junit4] 2> 1395162 T5636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/'
[junit4] 2> 1395185 T5636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/solr.xml
[junit4] 2> 1395207 T5636 oasc.CoreContainer.<init> New CoreContainer 1679240044
[junit4] 2> 1395207 T5636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/]
[junit4] 2> 1395208 T5636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1395208 T5636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1395208 T5636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1395208 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1395209 T5636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1395209 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1395209 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1395209 T5636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1395209 T5636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1395212 T5636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1395212 T5636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1395212 T5636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1395212 T5636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45432/solr
[junit4] 2> 1395213 T5636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1395213 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1395215 T5680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18cd7047 name:ZooKeeperConnection Watcher:127.0.0.1:45432 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1395215 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1395216 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1395216 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1395218 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1395219 T5682 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@339f9230 name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1395220 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1395242 T5636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1396247 T5636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49373_z
[junit4] 2> 1396249 T5636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49373_z
[junit4] 2> 1396264 T5683 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1396264 T5683 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1396265 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1396265 T5683 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1396266 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1396266 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1396268 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1396269 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49373/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49373_z",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1396270 T5661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 1396270 T5661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1396274 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1396377 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1396377 T5682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1396377 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397266 T5683 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1397266 T5683 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1397267 T5683 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1397268 T5683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1397269 T5683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1397270 T5683 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/'
[junit4] 2> 1397271 T5683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 1397271 T5683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 1397272 T5683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 1397305 T5683 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1397324 T5683 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1397326 T5683 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1397346 T5683 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1397515 T5683 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1397519 T5683 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1397520 T5683 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1397534 T5683 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1397537 T5683 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1397540 T5683 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1397541 T5683 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1397541 T5683 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1397542 T5683 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1397543 T5683 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1397543 T5683 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1397543 T5683 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1397543 T5683 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1397544 T5683 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/data/
[junit4] 2> 1397544 T5683 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3abea411
[junit4] 2> 1397545 T5683 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/data
[junit4] 2> 1397545 T5683 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/data/index/
[junit4] 2> 1397545 T5683 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1397545 T5683 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-003/collection1/data/index
[junit4] 2> 1397546 T5683 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=740508858, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1397547 T5683 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c0b157 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b3df746),segFN=segments_1,generation=1}
[junit4] 2> 1397547 T5683 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1397552 T5683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1397552 T5683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1397553 T5683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1397553 T5683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1397553 T5683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1397553 T5683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1397554 T5683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1397554 T5683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1397554 T5683 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1397555 T5683 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1397556 T5683 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1397556 T5683 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1397556 T5683 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1397557 T5683 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1397557 T5683 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1397558 T5683 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1397559 T5683 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1397562 T5683 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1397565 T5683 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1397565 T5683 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1397566 T5683 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=24.5615234375, floorSegmentMB=1.0751953125, forceMergeDeletesPctAllowed=17.926191470419468, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3957502255189558
[junit4] 2> 1397566 T5683 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c0b157 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b3df746),segFN=segments_1,generation=1}
[junit4] 2> 1397567 T5683 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1397567 T5683 oass.SolrIndexSearcher.<init> Opening Searcher@49756b4b[collection1] main
[junit4] 2> 1397567 T5683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1397568 T5683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1397569 T5683 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1397569 T5683 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1397569 T5683 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1397569 T5683 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1397570 T5683 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1397570 T5683 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1397570 T5683 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 1397570 T5683 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1397572 T5684 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49756b4b[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1397573 T5683 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1397574 T5687 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:49373/z collection:collection1 shard:shard2
[junit4] 2> 1397575 T5636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1397575 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1397575 T5687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1397586 T5687 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1397588 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397588 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397588 T5687 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1397589 T5687 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1397588 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C3761 name=collection1 org.apache.solr.core.SolrCore@40f1b92a url=https://127.0.0.1:49373/z/collection1 node=127.0.0.1:49373_z C3761_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:49373/z, core=collection1, node_name=127.0.0.1:49373_z}
[junit4] 2> 1397589 T5687 C3761 P49373 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:49373/z/collection1/
[junit4] 2> 1397589 T5687 C3761 P49373 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1397589 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1397589 T5687 C3761 P49373 oasc.SyncStrategy.syncToMe https://127.0.0.1:49373/z/collection1/ has no replicas
[junit4] 2> 1397590 T5687 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:49373/z/collection1/ shard2
[junit4] 2> 1397590 T5687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1397591 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397595 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397598 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397701 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397701 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397701 T5682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397745 T5687 oasc.ZkController.register We are https://127.0.0.1:49373/z/collection1/ and leader is https://127.0.0.1:49373/z/collection1/
[junit4] 2> 1397746 T5687 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:49373/z
[junit4] 2> 1397746 T5687 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1397746 T5687 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1397746 T5687 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1397747 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397748 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397748 T5687 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1397748 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397749 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1397750 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:49373/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49373_z",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1397752 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1397785 T5636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1397786 T5636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1397791 T5636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1397792 T5636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:57144
[junit4] 2> 1397794 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1397795 T5636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1397795 T5636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004
[junit4] 2> 1397795 T5636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/'
[junit4] 2> 1397813 T5636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/solr.xml
[junit4] 2> 1397833 T5636 oasc.CoreContainer.<init> New CoreContainer 631439030
[junit4] 2> 1397833 T5636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/]
[junit4] 2> 1397834 T5636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1397834 T5636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1397834 T5636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1397834 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1397835 T5636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1397835 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1397835 T5636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1397835 T5636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1397836 T5636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1397837 T5636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1397838 T5636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1397838 T5636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1397838 T5636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45432/solr
[junit4] 2> 1397838 T5636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1397839 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1397841 T5698 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4dbca817 name:ZooKeeperConnection Watcher:127.0.0.1:45432 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1397841 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1397844 T5636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1397845 T5700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e707058 name:ZooKeeperConnection Watcher:127.0.0.1:45432/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1397845 T5636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1397854 T5682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397854 T5636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1397854 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1397854 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1398865 T5636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57144_z
[junit4] 2> 1398867 T5636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57144_z
[junit4] 2> 1398883 T5701 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1398883 T5701 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1398886 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1398886 T5701 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1398886 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1398887 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1398889 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1398890 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:57144/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:57144_z",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1398890 T5661 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1398891 T5661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1398894 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1398997 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1398997 T5682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1398997 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1398997 T5700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1399886 T5701 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1399887 T5701 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1399888 T5701 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1399888 T5701 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1399889 T5701 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1399890 T5701 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/'
[junit4] 2> 1399891 T5701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 1399891 T5701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 1399892 T5701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 1399926 T5701 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1399945 T5701 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1399947 T5701 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1399951 T5701 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1400123 T5701 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1400127 T5701 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1400128 T5701 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1400138 T5701 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1400141 T5701 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1400142 T5701 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1400143 T5701 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1400143 T5701 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1400144 T5701 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1400144 T5701 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1400144 T5701 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1400145 T5701 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1400145 T5701 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1400145 T5701 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/data/
[junit4] 2> 1400145 T5701 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3abea411
[junit4] 2> 1400146 T5701 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/data
[junit4] 2> 1400146 T5701 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/data/index/
[junit4] 2> 1400146 T5701 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1400146 T5701 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-004/collection1/data/index
[junit4] 2> 1400147 T5701 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=740508858, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1400147 T5701 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@35b37592 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c0e6806),segFN=segments_1,generation=1}
[junit4] 2> 1400147 T5701 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1400151 T5701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1400152 T5701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1400152 T5701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1400152 T5701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1400152 T5701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1400152 T5701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1400153 T5701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1400153 T5701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1400153 T5701 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1400154 T5701 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1400154 T5701 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1400155 T5701 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1400155 T5701 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1400155 T5701 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1400156 T5701 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1400156 T5701 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1400157 T5701 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1400159 T5701 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1400162 T5701 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1400162 T5701 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1400162 T5701 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=24.5615234375, floorSegmentMB=1.0751953125, forceMergeDeletesPctAllowed=17.926191470419468, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3957502255189558
[junit4] 2> 1400163 T5701 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@35b37592 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c0e6806),segFN=segments_1,generation=1}
[junit4] 2> 1400163 T5701 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1400163 T5701 oass.SolrIndexSearcher.<init> Opening Searcher@4dbb5f2[collection1] main
[junit4] 2> 1400164 T5701 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1400164 T5701 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1400165 T5701 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1400165 T5701 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1400165 T5701 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1400166 T5701 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1400166 T5701 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1400166 T5701 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1400167 T5701 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 1400167 T5701 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1400169 T5702 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4dbb5f2[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1400172 T5701 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1400172 T5705 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:57144/z collection:collection1 shard:shard1
[junit4] 2> 1400173 T5636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1400173 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1400173 T5705 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1400180 T5705 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1400182 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400182 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400183 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400183 T5705 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1400183 T5705 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3762 name=collection1 org.apache.solr.core.SolrCore@41fc5f5a url=https://127.0.0.1:57144/z/collection1 node=127.0.0.1:57144_z C3762_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:57144/z, core=collection1, node_name=127.0.0.1:57144_z}
[junit4] 2> 1400184 T5705 C3762 P57144 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:57144/z/collection1/
[junit4] 2> 1400184 T5705 C3762 P57144 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1400184 T5705 C3762 P57144 oasc.SyncStrategy.syncToMe https://127.0.0.1:57144/z/collection1/ has no replicas
[junit4] 2> 1400184 T5705 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:57144/z/collection1/ shard1
[junit4] 2> 1400184 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1400184 T5705 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1400188 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400190 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400193 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400295 T5669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1400295 T5700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1400295 T5682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1400295 T5660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1400341 T5705 oasc.ZkController.register We are https://127.0.0.1:57144/z/collection1/ and leader is https://127.0.0.1:57144/z/collection1/
[junit4] 2> 1400342 T5705 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:57144/z
[junit4] 2> 1400342 T5705 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1400342 T5705 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1400342 T5705 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1400343 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400344 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400344 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400344 T5705 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1400345 T5661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1400345 T5661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:57144/z",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:57144_z",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 1400347 T5660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1400381 T5636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 1400381 T5636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1400395 T5636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1400396 T5636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:43923
[junit4] 2> 1400397 T5636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1400398 T5636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1400398 T5636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-005
[junit4] 2> 1400398 T5636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-005/'
[junit4] 2> 1400410 T5636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-005/solr.xml
[junit4] 2> 1400432 T5636 oasc.CoreContainer.<init> New CoreContainer 835826464
[junit4] 2> 1400432 T5636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-005/]
[junit4] 2> 1400433 T5636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1400433 T5636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2>
[...truncated too long message...]
(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1478756 T5718 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1478752 T5636 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 1478753 T5738 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1478757 T5738 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data;done=false>>]
[junit4] 2> 1478757 T5738 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data
[junit4] 2> 1478758 T5738 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data/index;done=false>>]
[junit4] 2> 1478758 T5738 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001/tempDir-006/collection1/data/index
[junit4] 2> 1478758 T5738 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1478759 T5738 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-DD151805B732301F-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=mk, timezone=Asia/Ashgabat
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=8,threads=1,free=174588096,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [PreAnalyzedUpdateProcessorTest, TermVectorComponentDistributedTest, DirectUpdateHandlerTest, TestWordDelimiterFilterFactory, ResourceLoaderTest, CoreAdminCreateDiscoverTest, TestRandomFaceting, DeleteReplicaTest, TestSolrXmlPersistence, SuggestComponentTest, TestSolrXMLSerializer, TestQuerySenderListener, TestSchemaNameResource, TestUniqueKeyFieldResource, TestSolrQueryParserDefaultOperatorResource, TestLRUCache, TestDocumentBuilder, DistributedQueryComponentOptimizationTest, TestSolrQueryParserResource, TestLMDirichletSimilarityFactory, TestMultiCoreConfBootstrap, CollectionsAPIDistributedZkTest, NoFacetTest, CoreContainerCoreInitFailuresTest, TestArbitraryIndexDir, TestXIncludeConfig, SolrCmdDistributorTest, RequestHandlersTest, ExpressionTest, SolrPluginUtilsTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFieldTypeCollectionResource, OverseerStatusTest, TestQueryTypes, TestQuerySenderNoQuery, BJQParserTest, HighlighterConfigTest, TestAddFieldRealTimeGet, OutputWriterTest, TestFileDictionaryLookup, SolrRequestParserTest, TestFastWriter, TestSchemaSimilarityResource, DocValuesMultiTest, TestDFRSimilarityFactory, StatsComponentTest, SpellCheckCollatorTest, TestCSVLoader, TestStressUserVersions, DisMaxRequestHandlerTest, IndexSchemaRuntimeFieldTest, ResponseHeaderTest, TestSolrJ, AbstractAnalyticsFacetTest, ShardRoutingCustomTest, NoCacheHeaderTest, SystemInfoHandlerTest, OpenExchangeRatesOrgProviderTest, PingRequestHandlerTest, CachingDirectoryFactoryTest, JSONWriterTest, TestCopyFieldCollectionResource, TestBinaryResponseWriter, DocExpirationUpdateProcessorFactoryTest, TestSweetSpotSimilarityFactory, HardAutoCommitTest, AbstractAnalyticsStatsTest, TestNoOpRegenerator, TestFiltering, QueryFacetTest, DefaultValueUpdateProcessorTest, TestAnalyzedSuggestions, TestIndexSearcher, InfoHandlerTest, TestFastOutputStream, TestCSVResponseWriter, ZkSolrClientTest, TestBinaryField, XsltUpdateRequestHandlerTest, TestSolrDeletionPolicy2, TestSolrCoreProperties, AnalysisErrorHandlingTest, HdfsBasicDistributedZkTest, FullSolrCloudDistribCmdsTest, TestStressRecovery, TestComplexPhraseQParserPlugin, TestCoreDiscovery, TestReload, TestSolrXmlPersistor, XmlUpdateRequestHandlerTest, DistanceFunctionTest, TestCursorMarkWithoutUniqueKey, TestRangeQuery, RequiredFieldsTest, AtomicUpdatesTest, DebugComponentTest, NotRequiredUniqueKeyTest, DirectSolrConnectionTest, OpenCloseCoreStressTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, TestZkChroot, ZkCLITest, AutoCommitTest, TestDistributedGrouping, TestHashPartitioner, TestStressReorder, TestStressVersions, TestCoreContainer, SolrCoreTest, TestGroupingSearch, BasicFunctionalityTest, OverseerCollectionProcessorTest, HighlighterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SuggesterTSTTest, TestStressLucene, SpatialFilterTest, SolrCoreCheckLockOnStartupTest, SortByFunctionTest, TestRemoteStreaming, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, DocumentBuilderTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, RegexBoostProcessorTest, TestCollationField, MBeansHandlerTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, SearchHandlerTest, CopyFieldTest, BadComponentTest, TestMergePolicyConfig, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, TestPhraseSuggestions, TestIBSimilarityFactory, LegacyHTMLStripCharFilterTest, URLClassifyProcessorTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, CustomCollectionTest, DeleteShardTest, MigrateRouteKeyTest, MultiThreadedOCPTest]
[junit4] Completed on J1 in 110.21s, 1 test, 1 error <<< FAILURES!
[...truncated 229 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 error, 36 ignored (15 assumptions)
Total time: 58 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build #
10511 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10511/
Java: 32bit/jdk1.8.0_05 -client -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 201 from https://127.0.0.1:51867/collection1lastClient and got 189 from https://127.0.0.1:34696/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 201 from https://127.0.0.1:51867/collection1lastClient and got 189 from https://127.0.0.1:34696/collection1
at __randomizedtesting.SeedInfo.seed([BEF162510CC9F334:3F17EC497B969308]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11238 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/init-core-data-001
[junit4] 2> 928677 T3636 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 928678 T3636 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 928681 T3636 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 928682 T3636 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 928682 T3637 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 928782 T3636 oasc.ZkTestServer.run start zk server on port:49063
[junit4] 2> 928783 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928832 T3643 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e3b8d7 name:ZooKeeperConnection Watcher:127.0.0.1:49063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928833 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928833 T3636 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 928837 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928838 T3645 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@386523 name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928838 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928839 T3636 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 928841 T3636 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 928843 T3636 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 928845 T3636 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 928847 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 928847 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 928850 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 928851 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 928853 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 928853 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 928855 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 928856 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 928857 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 928858 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 928859 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 928860 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 928862 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 928862 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 928863 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 928864 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 928865 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 928866 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 928868 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 928868 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 928870 T3636 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 928871 T3636 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 928874 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928875 T3647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17e1703 name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928876 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 929140 T3636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 929143 T3636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 929146 T3636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51116
[junit4] 2> 929147 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 929148 T3636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 929148 T3636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002
[junit4] 2> 929149 T3636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/'
[junit4] 2> 929162 T3636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/solr.xml
[junit4] 2> 929171 T3636 oasc.CoreContainer.<init> New CoreContainer 20737770
[junit4] 2> 929172 T3636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/]
[junit4] 2> 929172 T3636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 929173 T3636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 929173 T3636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 929173 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 929174 T3636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 929174 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 929174 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 929175 T3636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 929175 T3636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 929177 T3636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 929177 T3636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 929178 T3636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 929178 T3636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49063/solr
[junit4] 2> 929178 T3636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 929179 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 929180 T3658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b55a1 name:ZooKeeperConnection Watcher:127.0.0.1:49063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 929181 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 929183 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 929184 T3660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1636dc8 name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 929185 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 929186 T3636 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 929189 T3636 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 929191 T3636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 929194 T3636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 929197 T3636 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 929200 T3636 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 929202 T3636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51116_
[junit4] 2> 929203 T3636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51116_
[junit4] 2> 929205 T3636 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 929207 T3636 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 929210 T3636 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51116_
[junit4] 2> 929211 T3636 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 929213 T3636 oasc.Overseer.start Overseer (id=91899915051991044-127.0.0.1:51116_-n_0000000000) starting
[junit4] 2> 929215 T3636 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 929222 T3662 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 929222 T3636 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 929225 T3636 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 929226 T3636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 929228 T3661 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 929232 T3663 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 929232 T3663 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 929233 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929233 T3663 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 929234 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 929235 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51116",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51116_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 929235 T3661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 929235 T3661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 929237 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 929243 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 930234 T3663 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 930234 T3663 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 930235 T3663 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 930235 T3663 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 930236 T3663 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 930236 T3663 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/collection1/'
[junit4] 2> 930237 T3663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 930237 T3663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 930237 T3663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 930265 T3663 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 930276 T3663 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 930278 T3663 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 930285 T3663 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 930394 T3663 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 930395 T3663 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 930396 T3663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930399 T3663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930413 T3663 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 930414 T3663 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 930414 T3663 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/
[junit4] 2> 930414 T3663 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@117178c
[junit4] 2> 930415 T3663 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data
[junit4] 2> 930415 T3663 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/index/
[junit4] 2> 930416 T3663 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 930416 T3663 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/index
[junit4] 2> 930417 T3663 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=92.580078125, floorSegmentMB=1.53125, forceMergeDeletesPctAllowed=8.828951119659278, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6057923387262849
[junit4] 2> 930422 T3663 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 930423 T3663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 930425 T3663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 930426 T3663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 930426 T3663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 930426 T3663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 930426 T3663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 930427 T3663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 930427 T3663 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 930427 T3663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 930427 T3663 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 930428 T3663 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 930428 T3663 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 930429 T3663 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 930429 T3663 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 930429 T3663 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 930429 T3663 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 930430 T3663 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 930430 T3663 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 930432 T3663 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 930433 T3663 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 930433 T3663 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 930434 T3663 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=80.87890625, floorSegmentMB=0.728515625, forceMergeDeletesPctAllowed=7.3048084134437286, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 930435 T3663 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 930435 T3663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 930436 T3663 oass.SolrIndexSearcher.<init> Opening Searcher@278995[collection1] main
[junit4] 2> 930436 T3663 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 930437 T3663 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 930437 T3663 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 930438 T3663 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 930438 T3663 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 930438 T3663 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 930438 T3663 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 930439 T3663 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 930439 T3663 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 930439 T3663 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 930442 T3664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@278995[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 930443 T3663 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 930444 T3667 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:51116 collection:control_collection shard:shard1
[junit4] 2> 930444 T3636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 930445 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 930445 T3667 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 930447 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 930448 T3669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@511d1c name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 930449 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 930450 T3636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 930450 T3667 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 930451 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930451 T3636 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 930452 T3667 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 930452 T3667 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2693 name=collection1 org.apache.solr.core.SolrCore@1f32a76 url=https://127.0.0.1:51116/collection1 node=127.0.0.1:51116_ C2693_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:51116, core=collection1, node_name=127.0.0.1:51116_}
[junit4] 2> 930452 T3667 C2693 P51116 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:51116/collection1/
[junit4] 2> 930453 T3667 C2693 P51116 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 930453 T3667 C2693 P51116 oasc.SyncStrategy.syncToMe https://127.0.0.1:51116/collection1/ has no replicas
[junit4] 2> 930453 T3667 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:51116/collection1/ shard1
[junit4] 2> 930454 T3667 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 930454 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 930456 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930458 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930460 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930562 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 930562 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 930609 T3667 oasc.ZkController.register We are https://127.0.0.1:51116/collection1/ and leader is https://127.0.0.1:51116/collection1/
[junit4] 2> 930609 T3667 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51116
[junit4] 2> 930609 T3667 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 930610 T3667 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 930610 T3667 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 930611 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930612 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930612 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930612 T3667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 930613 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 930613 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:51116",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51116_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 930622 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 930724 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 930724 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 930740 T3636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 930741 T3636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 930744 T3636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 930745 T3636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:48159
[junit4] 2> 930747 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 930748 T3636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 930749 T3636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003
[junit4] 2> 930749 T3636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/'
[junit4] 2> 930769 T3636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/solr.xml
[junit4] 2> 930793 T3636 oasc.CoreContainer.<init> New CoreContainer 17496519
[junit4] 2> 930793 T3636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/]
[junit4] 2> 930794 T3636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 930794 T3636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 930794 T3636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 930795 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 930795 T3636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 930795 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 930796 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 930796 T3636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 930796 T3636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 930798 T3636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 930799 T3636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 930799 T3636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 930799 T3636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49063/solr
[junit4] 2> 930800 T3636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 930801 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 930802 T3680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dea563 name:ZooKeeperConnection Watcher:127.0.0.1:49063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 930802 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 930804 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 930805 T3682 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1454da name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 930806 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 930812 T3636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 931814 T3636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48159_
[junit4] 2> 931815 T3636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48159_
[junit4] 2> 931825 T3683 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 931825 T3683 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 931826 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931826 T3683 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 931826 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931827 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931828 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 931829 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:48159",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48159_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 931829 T3661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 931829 T3661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 931831 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 931932 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931932 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931932 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 932827 T3683 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 932827 T3683 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 932828 T3683 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 932828 T3683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 932829 T3683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 932829 T3683 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/collection1/'
[junit4] 2> 932830 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 932830 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 932831 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 932853 T3683 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 932864 T3683 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 932865 T3683 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 932870 T3683 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 932976 T3683 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 932978 T3683 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 932979 T3683 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 932982 T3683 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 932996 T3683 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 932997 T3683 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 932997 T3683 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/
[junit4] 2> 932997 T3683 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@117178c
[junit4] 2> 932998 T3683 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1
[junit4] 2> 932998 T3683 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/index/
[junit4] 2> 932999 T3683 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 932999 T3683 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/index
[junit4] 2> 933000 T3683 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=92.580078125, floorSegmentMB=1.53125, forceMergeDeletesPctAllowed=8.828951119659278, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6057923387262849
[junit4] 2> 933009 T3683 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 933009 T3683 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 933013 T3683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 933013 T3683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 933013 T3683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 933013 T3683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 933014 T3683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 933014 T3683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 933014 T3683 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 933014 T3683 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 933015 T3683 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 933015 T3683 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 933016 T3683 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 933016 T3683 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 933016 T3683 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 933017 T3683 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 933017 T3683 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 933018 T3683 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 933018 T3683 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 933020 T3683 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 933021 T3683 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 933022 T3683 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 933022 T3683 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=80.87890625, floorSegmentMB=0.728515625, forceMergeDeletesPctAllowed=7.3048084134437286, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 933023 T3683 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 933024 T3683 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 933024 T3683 oass.SolrIndexSearcher.<init> Opening Searcher@16e6aad[collection1] main
[junit4] 2> 933024 T3683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 933025 T3683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 933025 T3683 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 933025 T3683 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 933026 T3683 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 933026 T3683 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 933026 T3683 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 933027 T3683 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 933027 T3683 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 933027 T3683 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 933032 T3684 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16e6aad[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 933035 T3683 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 933035 T3687 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:48159 collection:collection1 shard:shard2
[junit4] 2> 933036 T3636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 933036 T3687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 933036 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 933042 T3687 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 933045 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933045 T3687 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 933045 T3687 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 933045 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C2694 name=collection1 org.apache.solr.core.SolrCore@83c6a9 url=https://127.0.0.1:48159/collection1 node=127.0.0.1:48159_ C2694_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:48159, core=collection1, node_name=127.0.0.1:48159_}
[junit4] 2> 933045 T3687 C2694 P48159 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:48159/collection1/
[junit4] 2> 933046 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933046 T3687 C2694 P48159 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 933046 T3687 C2694 P48159 oasc.SyncStrategy.syncToMe https://127.0.0.1:48159/collection1/ has no replicas
[junit4] 2> 933047 T3687 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:48159/collection1/ shard2
[junit4] 2> 933046 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933047 T3687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 933049 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933051 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933053 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933155 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933155 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933155 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933202 T3687 oasc.ZkController.register We are https://127.0.0.1:48159/collection1/ and leader is https://127.0.0.1:48159/collection1/
[junit4] 2> 933203 T3687 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:48159
[junit4] 2> 933203 T3687 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 933203 T3687 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 933204 T3687 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 933205 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933205 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933205 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933205 T3687 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933206 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 933207 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:48159",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48159_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 933209 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 933316 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933316 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933316 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 933333 T3636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 933334 T3636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 933336 T3636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 933337 T3636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:60437
[junit4] 2> 933340 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 933340 T3636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 933340 T3636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004
[junit4] 2> 933341 T3636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/'
[junit4] 2> 933354 T3636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/solr.xml
[junit4] 2> 933364 T3636 oasc.CoreContainer.<init> New CoreContainer 26869278
[junit4] 2> 933364 T3636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/]
[junit4] 2> 933365 T3636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 933365 T3636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 933365 T3636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 933366 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 933366 T3636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 933366 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 933367 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 933367 T3636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 933367 T3636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 933369 T3636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 933370 T3636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 933370 T3636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 933370 T3636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49063/solr
[junit4] 2> 933371 T3636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 933371 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 933373 T3698 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@195608f name:ZooKeeperConnection Watcher:127.0.0.1:49063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 933373 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 933375 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 933376 T3700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19bac39 name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 933376 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 933381 T3636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 934383 T3636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60437_
[junit4] 2> 934385 T3636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60437_
[junit4] 2> 934391 T3701 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 934392 T3701 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 934392 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 934392 T3701 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 934393 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 934393 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 934394 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 934395 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:60437",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60437_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 934395 T3661 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 934395 T3661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 934397 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 934499 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934499 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934499 T3700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934499 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935393 T3701 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 935393 T3701 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 935394 T3701 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 935394 T3701 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 935395 T3701 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 935395 T3701 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/collection1/'
[junit4] 2> 935396 T3701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 935396 T3701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 935396 T3701 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 935420 T3701 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 935431 T3701 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 935431 T3701 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 935437 T3701 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 935544 T3701 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 935545 T3701 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 935546 T3701 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 935549 T3701 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 935562 T3701 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 935562 T3701 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 935563 T3701 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/
[junit4] 2> 935563 T3701 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@117178c
[junit4] 2> 935563 T3701 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2
[junit4] 2> 935564 T3701 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/index/
[junit4] 2> 935564 T3701 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 935565 T3701 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/index
[junit4] 2> 935565 T3701 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=92.580078125, floorSegmentMB=1.53125, forceMergeDeletesPctAllowed=8.828951119659278, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6057923387262849
[junit4] 2> 935572 T3701 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 935572 T3701 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 935575 T3701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 935576 T3701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 935576 T3701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 935576 T3701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 935576 T3701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 935576 T3701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 935577 T3701 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 935577 T3701 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 935577 T3701 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 935578 T3701 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 935579 T3701 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 935579 T3701 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 935579 T3701 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 935580 T3701 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 935580 T3701 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 935581 T3701 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 935581 T3701 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 935583 T3701 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 935584 T3701 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 935584 T3701 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 935585 T3701 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=80.87890625, floorSegmentMB=0.728515625, forceMergeDeletesPctAllowed=7.3048084134437286, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 935586 T3701 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 935586 T3701 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 935586 T3701 oass.SolrIndexSearcher.<init> Opening Searcher@e5a1d7[collection1] main
[junit4] 2> 935587 T3701 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 935587 T3701 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 935588 T3701 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 935588 T3701 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 935588 T3701 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 935588 T3701 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 935589 T3701 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 935589 T3701 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 935589 T3701 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 935590 T3701 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 935593 T3702 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e5a1d7[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 935597 T3701 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 935597 T3705 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:60437 collection:collection1 shard:shard3
[junit4] 2> 935598 T3636 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 935598 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 935598 T3705 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 935606 T3705 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 935607 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935607 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935608 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935608 T3705 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 935608 T3705 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2695 name=collection1 org.apache.solr.core.SolrCore@3fee31 url=https://127.0.0.1:60437/collection1 node=127.0.0.1:60437_ C2695_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:60437, core=collection1, node_name=127.0.0.1:60437_}
[junit4] 2> 935609 T3705 C2695 P60437 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:60437/collection1/
[junit4] 2> 935609 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 935609 T3705 C2695 P60437 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 935609 T3705 C2695 P60437 oasc.SyncStrategy.syncToMe https://127.0.0.1:60437/collection1/ has no replicas
[junit4] 2> 935610 T3705 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:60437/collection1/ shard3
[junit4] 2> 935610 T3705 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 935612 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935615 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935617 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935719 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935719 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935719 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935719 T3700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935766 T3705 oasc.ZkController.register We are https://127.0.0.1:60437/collection1/ and leader is https://127.0.0.1:60437/collection1/
[junit4] 2> 935767 T3705 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:60437
[junit4] 2> 935767 T3705 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 935767 T3705 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 935767 T3705 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 935768 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935769 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935769 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935769 T3705 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 935770 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 935771 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:60437",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60437_",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 935775 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 935877 T3669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935877 T3700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935877 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935877 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935879 T3636 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 935879 T3636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 935882 T3636 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 935883 T3636 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:36488
[junit4] 2> 935885 T3636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 935885 T3636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 935885 T3636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-005
[junit4] 2> 935886 T3636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-005/'
[junit4] 2> 935899 T3636 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-005/solr.xml
[junit4] 2> 935923 T3636 oasc.CoreContainer.<init> New CoreContainer 10259546
[junit4] 2> 935924 T3636 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-005/]
[junit4] 2> 935924 T3636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 935925 T3636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 935925 T3636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 935925 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 935925 T3636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 935926 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 935926 T3636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 935926 T3636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 935927 T3636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 935929 T3636 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 935929 T3636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 935929 T3636 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 935930 T3636 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49063/solr
[junit4] 2> 935930 T3636 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 935931 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 935932 T3716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f4554 name:ZooKeeperConnection Watcher:127.0.0.1:49063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 935933 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 935934 T3636 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 935935 T3718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c32c86 name:ZooKeeperConnection Watcher:127.0.0.1:49063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 935936 T3636 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 935941 T3636 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 936945 T3636 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36488_
[junit4] 2> 936947 T3636 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36488_
[junit4] 2> 936952 T3719 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 936952 T3719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 936953 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 936953 T3719 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 936953 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 936954 T3660 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 936954 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 936955 T3661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36488",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:36488_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 936955 T3661 oas
[...truncated too long message...]
[junit4] 2> 1075306 T4238 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12;done=false>>]
[junit4] 2> 1075307 T4238 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12
[junit4] 2> 1075307 T4238 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12/index.20140609033752297 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12/index.20140609033752297;done=false>>]
[junit4] 2> 1075307 T4238 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001/tempDir-001/jetty12/index.20140609033752297
[junit4] 2> 1075307 T4238 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:243)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:346)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:330)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:190)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1075308 T4238 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1075308 T4238 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BEF162510CC9F334-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=en_CA, timezone=Africa/Brazzaville
[junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_05 (32-bit)/cpus=8,threads=1,free=236314768,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [UnloadDistributedZkTest, JsonLoaderTest, DocValuesTest, TestGroupingSearch, TestSort, SynonymTokenizerTest, ZkCLITest, TestSearchPerf, HttpPartitionTest, TestCoreContainer, TermsComponentTest, OverseerStatusTest, RegexBoostProcessorTest, CopyFieldTest, TestReloadAndDeleteDocs, QueryElevationComponentTest, MoreLikeThisHandlerTest, BlockCacheTest, TestSolrIndexConfig, TestStressUserVersions, TestJoin, TestRealTimeGet, CursorPagingTest, TestModifyConfFiles, IndexSchemaTest, OverseerTest, TestRemoteStreaming, RecoveryZkTest, TestDocSet, QueryFacetTest, TermVectorComponentTest, CircularListTest, TestConfig, SpellingQueryConverterTest, TestFaceting, CoreAdminHandlerTest, TestStandardQParsers, AutoCommitTest, ModifyConfFileTest, TestOverriddenPrefixQueryForCustomFieldType, SearchHandlerTest, TestCopyFieldCollectionResource, PreAnalyzedFieldTest, SuggesterTest, DirectUpdateHandlerOptimizeTest, DocumentBuilderTest, SliceStateUpdateTest, SolrCoreTest, StressHdfsTest, SyncSliceTest, UpdateParamsTest, TestCursorMarkWithoutUniqueKey, DistributedQueryComponentOptimizationTest, TestSimpleQParserPlugin, TestPostingsSolrHighlighter, TestDistributedMissingSort, TestBlendedInfixSuggestions, PolyFieldTest, HdfsUnloadDistributedZkTest, TestStressVersions, TestRandomDVFaceting, QueryResultKeyTest, TestShortCircuitedRequests, TestCollectionAPI, EchoParamsTest, TimeZoneUtilsTest, TestPseudoReturnFields, TestCollationFieldDocValues, TestFieldTypeResource, AnalysisAfterCoreReloadTest, TestValueSourceCache, SimpleFacetsTest, TestPHPSerializedResponseWriter, StatelessScriptUpdateProcessorFactoryTest, AddBlockUpdateTest, QueryParsingTest, DistributedQueryElevationComponentTest, PrimitiveFieldTypeTest, TestNonDefinedSimilarityFactory, DeleteShardTest, SuggesterFSTTest, TestNonNRTOpen, ScriptEngineTest, RAMDirectoryFactoryTest, ExpressionTest, CSVRequestHandlerTest, FastVectorHighlighterTest, FieldMutatingUpdateProcessorTest, OverseerCollectionProcessorTest, ClusterStateUpdateTest, SolrInfoMBeanTest, TestComplexPhraseQParserPlugin, AssignTest, URLClassifyProcessorTest, ZkControllerTest, DateRangeFieldTest, ClusterStateTest, DistribCursorPagingTest, TestRestManager, SchemaVersionSpecificBehaviorTest, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J0 in 146.92s, 1 test, 1 failure <<< FAILURES!
[...truncated 622 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 failure, 36 ignored (15 assumptions)
Total time: 65 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_05 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b15) -
Build # 10510 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10510/
Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 437 from http://127.0.0.1:33462/y_/aj/collection1lastClient and got 422 from http://127.0.0.1:44081/y_/aj/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 437 from http://127.0.0.1:33462/y_/aj/collection1lastClient and got 422 from http://127.0.0.1:44081/y_/aj/collection1
at __randomizedtesting.SeedInfo.seed([22ACEAC452226E54:A34A64DC257D0E68]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11201 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/init-core-data-001
[junit4] 2> 896963 T3499 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 896963 T3499 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /y_/aj
[junit4] 2> 896966 T3499 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 896967 T3499 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 896967 T3500 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 897067 T3499 oasc.ZkTestServer.run start zk server on port:53210
[junit4] 2> 897068 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 897146 T3506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a60087f name:ZooKeeperConnection Watcher:127.0.0.1:53210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 897146 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 897147 T3499 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 897153 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 897155 T3508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b2588d5 name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 897155 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 897156 T3499 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 897160 T3499 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 897162 T3499 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 897164 T3499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 897165 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 897166 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 897170 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 897171 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 897174 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 897175 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 897177 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 897178 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 897181 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 897181 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 897184 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 897184 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 897187 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 897188 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 897191 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 897191 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 897194 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 897195 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 897198 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 897199 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 897200 T3499 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 897201 T3499 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 897438 T3499 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 897441 T3499 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45091
[junit4] 2> 897442 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 897443 T3499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 897443 T3499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002
[junit4] 2> 897444 T3499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/'
[junit4] 2> 897466 T3499 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/solr.xml
[junit4] 2> 897480 T3499 oasc.CoreContainer.<init> New CoreContainer 456529681
[junit4] 2> 897480 T3499 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/]
[junit4] 2> 897481 T3499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 897481 T3499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 897481 T3499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 897481 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 897481 T3499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 897482 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 897482 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 897482 T3499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 897482 T3499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 897483 T3499 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 897483 T3499 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 897483 T3499 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 897484 T3499 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53210/solr
[junit4] 2> 897484 T3499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 897485 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 897487 T3519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7db5c24a name:ZooKeeperConnection Watcher:127.0.0.1:53210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 897487 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 897490 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 897491 T3521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5436462d name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 897492 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 897493 T3499 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 897495 T3499 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 897496 T3499 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 897498 T3499 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 897500 T3499 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 897502 T3499 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 897503 T3499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45091_y_%2Faj
[junit4] 2> 897505 T3499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45091_y_%2Faj
[junit4] 2> 897508 T3499 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 897510 T3499 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 897512 T3499 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45091_y_%2Faj
[junit4] 2> 897512 T3499 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 897513 T3499 oasc.Overseer.start Overseer (id=91899395886481411-127.0.0.1:45091_y_%2Faj-n_0000000000) starting
[junit4] 2> 897515 T3499 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 897521 T3523 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 897521 T3499 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 897523 T3499 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 897524 T3499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 897525 T3522 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 897526 T3524 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 897526 T3524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 897527 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 897527 T3524 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 897527 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 897528 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45091/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45091_y_%2Faj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 897528 T3522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 897528 T3522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 897529 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 897540 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 898527 T3524 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 898528 T3524 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 898529 T3524 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 898529 T3524 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 898530 T3524 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 898531 T3524 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/collection1/'
[junit4] 2> 898531 T3524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 898532 T3524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 898532 T3524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 898551 T3524 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 898559 T3524 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 898560 T3524 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 898564 T3524 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 898661 T3524 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 898662 T3524 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 898663 T3524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 898665 T3524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 898675 T3524 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 898675 T3524 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 898682 T3524 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/
[junit4] 2> 898682 T3524 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b177630
[junit4] 2> 898682 T3524 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data
[junit4] 2> 898683 T3524 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/index/
[junit4] 2> 898683 T3524 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 898683 T3524 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/index
[junit4] 2> 898683 T3524 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=64.849609375, floorSegmentMB=1.6953125, forceMergeDeletesPctAllowed=18.807660060495177, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 898689 T3524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 898689 T3524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 898691 T3524 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 898691 T3524 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 898692 T3524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 898693 T3524 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 898693 T3524 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 898693 T3524 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 898693 T3524 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 898693 T3524 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 898693 T3524 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 898694 T3524 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 898694 T3524 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 898694 T3524 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 898696 T3524 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 898698 T3524 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 898698 T3524 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 898699 T3524 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=37.748046875, floorSegmentMB=0.953125, forceMergeDeletesPctAllowed=29.960859252724052, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5706901075809272
[junit4] 2> 898700 T3524 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 898701 T3524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 898701 T3524 oass.SolrIndexSearcher.<init> Opening Searcher@6c7dba2c[collection1] main
[junit4] 2> 898701 T3524 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 898702 T3524 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 898702 T3524 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 898703 T3524 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 898703 T3524 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 898704 T3524 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 898705 T3524 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 898705 T3524 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 898706 T3524 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 898706 T3524 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 898708 T3525 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c7dba2c[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 898709 T3524 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 898710 T3528 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45091/y_/aj collection:control_collection shard:shard1
[junit4] 2> 898710 T3499 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 898711 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 898711 T3528 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 898712 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 898713 T3530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7078b490 name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 898713 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 898714 T3499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 898715 T3499 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 898716 T3528 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 898718 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898718 T3528 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 898718 T3528 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1673 name=collection1 org.apache.solr.core.SolrCore@4cab35e6 url=http://127.0.0.1:45091/y_/aj/collection1 node=127.0.0.1:45091_y_%2Faj C1673_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:45091/y_/aj, core=collection1, node_name=127.0.0.1:45091_y_%2Faj}
[junit4] 2> 898718 T3528 C1673 P45091 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45091/y_/aj/collection1/
[junit4] 2> 898718 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 898718 T3528 C1673 P45091 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 898719 T3528 C1673 P45091 oasc.SyncStrategy.syncToMe http://127.0.0.1:45091/y_/aj/collection1/ has no replicas
[junit4] 2> 898719 T3528 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45091/y_/aj/collection1/ shard1
[junit4] 2> 898719 T3528 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 898720 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898721 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898722 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898826 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 898826 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 898872 T3528 oasc.ZkController.register We are http://127.0.0.1:45091/y_/aj/collection1/ and leader is http://127.0.0.1:45091/y_/aj/collection1/
[junit4] 2> 898872 T3528 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45091/y_/aj
[junit4] 2> 898872 T3528 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 898872 T3528 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 898872 T3528 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 898874 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898874 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898874 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898874 T3528 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 898875 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 898875 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:45091/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45091_y_%2Faj",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 898876 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 898941 T3499 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 898942 T3499 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 898944 T3499 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41618
[junit4] 2> 898945 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 898946 T3499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 898946 T3499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003
[junit4] 2> 898946 T3499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/'
[junit4] 2> 898960 T3499 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/solr.xml
[junit4] 2> 898981 T3499 oasc.CoreContainer.<init> New CoreContainer 1560794574
[junit4] 2> 898981 T3499 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/]
[junit4] 2> 898982 T3499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 898983 T3499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 898983 T3499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 898984 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 898984 T3499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 898984 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 898985 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 898985 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 898985 T3499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 898986 T3499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 898985 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 898987 T3499 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 898987 T3499 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 898987 T3499 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 898988 T3499 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53210/solr
[junit4] 2> 898988 T3499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 898989 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 898991 T3541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bc98b92 name:ZooKeeperConnection Watcher:127.0.0.1:53210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 898991 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 898993 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 898995 T3543 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@428af993 name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 898995 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 898999 T3499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 900001 T3499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41618_y_%2Faj
[junit4] 2> 900002 T3499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41618_y_%2Faj
[junit4] 2> 900008 T3544 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 900009 T3544 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 900009 T3544 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 900009 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 900010 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 900010 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 900011 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 900011 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41618/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41618_y_%2Faj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 900011 T3522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 900012 T3522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 900013 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 900114 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 900114 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 900114 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901010 T3544 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 901010 T3544 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 901011 T3544 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 901011 T3544 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 901013 T3544 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 901013 T3544 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/collection1/'
[junit4] 2> 901014 T3544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 901015 T3544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 901015 T3544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 901045 T3544 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 901059 T3544 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 901060 T3544 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 901064 T3544 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 901183 T3544 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 901184 T3544 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 901185 T3544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 901187 T3544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 901202 T3544 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 901203 T3544 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 901203 T3544 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/
[junit4] 2> 901203 T3544 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b177630
[junit4] 2> 901203 T3544 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1
[junit4] 2> 901204 T3544 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/index/
[junit4] 2> 901204 T3544 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 901204 T3544 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/index
[junit4] 2> 901204 T3544 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=64.849609375, floorSegmentMB=1.6953125, forceMergeDeletesPctAllowed=18.807660060495177, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 901210 T3544 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 901211 T3544 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 901216 T3544 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 901216 T3544 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 901217 T3544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 901217 T3544 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 901217 T3544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 901218 T3544 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 901218 T3544 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 901219 T3544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 901219 T3544 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 901220 T3544 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 901220 T3544 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 901221 T3544 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 901221 T3544 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 901222 T3544 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 901222 T3544 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 901223 T3544 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 901223 T3544 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 901226 T3544 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 901228 T3544 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 901228 T3544 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 901229 T3544 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=37.748046875, floorSegmentMB=0.953125, forceMergeDeletesPctAllowed=29.960859252724052, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5706901075809272
[junit4] 2> 901231 T3544 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 901231 T3544 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 901231 T3544 oass.SolrIndexSearcher.<init> Opening Searcher@5eb72820[collection1] main
[junit4] 2> 901232 T3544 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 901233 T3544 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 901233 T3544 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 901234 T3544 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 901234 T3544 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 901235 T3544 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 901236 T3544 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 901236 T3544 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 901236 T3544 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 901236 T3544 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 901240 T3545 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5eb72820[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 901243 T3544 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 901244 T3548 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41618/y_/aj collection:collection1 shard:shard2
[junit4] 2> 901248 T3499 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 901248 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 901249 T3548 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 901255 T3548 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 901258 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901258 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901258 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901258 T3548 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 901259 T3548 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1674 name=collection1 org.apache.solr.core.SolrCore@6f6545e2 url=http://127.0.0.1:41618/y_/aj/collection1 node=127.0.0.1:41618_y_%2Faj C1674_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:41618/y_/aj, core=collection1, node_name=127.0.0.1:41618_y_%2Faj}
[junit4] 2> 901259 T3548 C1674 P41618 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41618/y_/aj/collection1/
[junit4] 2> 901259 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 901259 T3548 C1674 P41618 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 901259 T3548 C1674 P41618 oasc.SyncStrategy.syncToMe http://127.0.0.1:41618/y_/aj/collection1/ has no replicas
[junit4] 2> 901259 T3548 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41618/y_/aj/collection1/ shard2
[junit4] 2> 901260 T3548 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 901260 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901262 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901263 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901365 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901365 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901365 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901413 T3548 oasc.ZkController.register We are http://127.0.0.1:41618/y_/aj/collection1/ and leader is http://127.0.0.1:41618/y_/aj/collection1/
[junit4] 2> 901413 T3548 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41618/y_/aj
[junit4] 2> 901413 T3548 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 901413 T3548 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 901413 T3548 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 901414 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901414 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901414 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901415 T3548 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 901415 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 901416 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:41618/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41618_y_%2Faj",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 901417 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 901475 T3499 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 901476 T3499 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 901478 T3499 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33462
[junit4] 2> 901480 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 901480 T3499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 901481 T3499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004
[junit4] 2> 901481 T3499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/'
[junit4] 2> 901499 T3499 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/solr.xml
[junit4] 2> 901506 T3499 oasc.CoreContainer.<init> New CoreContainer 1174544407
[junit4] 2> 901507 T3499 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/]
[junit4] 2> 901507 T3499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 901507 T3499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 901508 T3499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 901508 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 901508 T3499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 901508 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 901508 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 901509 T3499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 901509 T3499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 901509 T3499 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 901510 T3499 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 901510 T3499 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 901510 T3499 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53210/solr
[junit4] 2> 901510 T3499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 901511 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 901513 T3559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d37982 name:ZooKeeperConnection Watcher:127.0.0.1:53210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 901514 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 901517 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 901518 T3561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f23b3b6 name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 901519 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 901557 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901557 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901557 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 901562 T3499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 902564 T3499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33462_y_%2Faj
[junit4] 2> 902565 T3499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33462_y_%2Faj
[junit4] 2> 902571 T3562 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 902571 T3562 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 902572 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 902572 T3562 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 902572 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 902572 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 902573 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 902573 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33462/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33462_y_%2Faj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 902573 T3522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 902574 T3522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 902575 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 902677 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 902677 T3561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 902677 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 902677 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 903572 T3562 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 903572 T3562 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 903573 T3562 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 903573 T3562 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 903574 T3562 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 903574 T3562 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/collection1/'
[junit4] 2> 903575 T3562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 903575 T3562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 903575 T3562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 903594 T3562 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 903614 T3562 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 903615 T3562 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 903619 T3562 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 903708 T3562 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 903709 T3562 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 903710 T3562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 903712 T3562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 903722 T3562 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 903722 T3562 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 903723 T3562 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/
[junit4] 2> 903723 T3562 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b177630
[junit4] 2> 903723 T3562 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2
[junit4] 2> 903723 T3562 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/index/
[junit4] 2> 903724 T3562 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 903724 T3562 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/index
[junit4] 2> 903724 T3562 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=64.849609375, floorSegmentMB=1.6953125, forceMergeDeletesPctAllowed=18.807660060495177, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 903730 T3562 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 903730 T3562 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 903733 T3562 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 903734 T3562 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 903734 T3562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 903734 T3562 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 903734 T3562 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 903735 T3562 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 903735 T3562 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 903735 T3562 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 903735 T3562 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 903735 T3562 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 903736 T3562 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 903736 T3562 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 903737 T3562 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 903738 T3562 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 903738 T3562 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 903739 T3562 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=37.748046875, floorSegmentMB=0.953125, forceMergeDeletesPctAllowed=29.960859252724052, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5706901075809272
[junit4] 2> 903740 T3562 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 903740 T3562 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 903740 T3562 oass.SolrIndexSearcher.<init> Opening Searcher@3bec006d[collection1] main
[junit4] 2> 903740 T3562 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 903741 T3562 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 903741 T3562 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 903741 T3562 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 903741 T3562 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 903741 T3562 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 903742 T3562 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 903742 T3562 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 903742 T3562 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 903742 T3562 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 903744 T3563 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3bec006d[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 903754 T3562 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 903755 T3566 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33462/y_/aj collection:collection1 shard:shard3
[junit4] 2> 903755 T3499 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 903755 T3566 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 903755 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 903759 T3566 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 903760 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903760 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903760 T3566 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 903761 T3566 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 903760 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C1675 name=collection1 org.apache.solr.core.SolrCore@14543171 url=http://127.0.0.1:33462/y_/aj/collection1 node=127.0.0.1:33462_y_%2Faj C1675_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:33462/y_/aj, core=collection1, node_name=127.0.0.1:33462_y_%2Faj}
[junit4] 2> 903761 T3566 C1675 P33462 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33462/y_/aj/collection1/
[junit4] 2> 903761 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 903761 T3566 C1675 P33462 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 903761 T3566 C1675 P33462 oasc.SyncStrategy.syncToMe http://127.0.0.1:33462/y_/aj/collection1/ has no replicas
[junit4] 2> 903762 T3566 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33462/y_/aj/collection1/ shard3
[junit4] 2> 903762 T3566 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 903763 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903764 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903766 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903867 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 903867 T3561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 903867 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 903867 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 903915 T3566 oasc.ZkController.register We are http://127.0.0.1:33462/y_/aj/collection1/ and leader is http://127.0.0.1:33462/y_/aj/collection1/
[junit4] 2> 903915 T3566 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33462/y_/aj
[junit4] 2> 903915 T3566 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 903915 T3566 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 903915 T3566 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 903916 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903917 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903917 T3566 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 903917 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903918 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 903918 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:33462/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33462_y_%2Faj",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 903923 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 903978 T3499 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 903979 T3499 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 903980 T3499 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60630
[junit4] 2> 903981 T3499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 903981 T3499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 903982 T3499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-005
[junit4] 2> 903982 T3499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-005/'
[junit4] 2> 903991 T3499 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-005/solr.xml
[junit4] 2> 903998 T3499 oasc.CoreContainer.<init> New CoreContainer 109307600
[junit4] 2> 903999 T3499 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-005/]
[junit4] 2> 903999 T3499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 903999 T3499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 903999 T3499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 904000 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 904000 T3499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 904000 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 904000 T3499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 904000 T3499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 904001 T3499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 904001 T3499 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 904001 T3499 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 904002 T3499 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 904002 T3499 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53210/solr
[junit4] 2> 904002 T3499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 904003 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 904008 T3577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c50b5d7 name:ZooKeeperConnection Watcher:127.0.0.1:53210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 904008 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 904010 T3499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 904011 T3579 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b098abf name:ZooKeeperConnection Watcher:127.0.0.1:53210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 904011 T3499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 904019 T3499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 904024 T3521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 904024 T3543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 904024 T3530 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 904024 T3561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 904024 T3579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 905022 T3499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60630_y_%2Faj
[junit4] 2> 905023 T3499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60630_y_%2Faj
[junit4] 2> 905029 T3580 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 905029 T3580 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 905030 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 905031 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 905031 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 905030 T3580 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 905032 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 905033 T3522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60630/y_/aj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60630_y_%2Faj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 905033 T3522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 905034 T3522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 905037 T3521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type Nod
[...truncated too long message...]
tion1lastClient and got 422 from http://127.0.0.1:44081/y_/aj/collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([22ACEAC452226E54:A34A64DC257D0E68]:0)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1131)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1110)
[junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1040222 T3499 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 1040991 T4784 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@dffe29a
[junit4] 2> 1041003 T4784 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=3,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=12646,transaction_logs_total_number=3}
[junit4] 2> 1041003 T4784 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1041003 T4784 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1041003 T4784 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1041006 T4784 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1041007 T4784 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1041007 T4784 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10;done=false>>]
[junit4] 2> 1041007 T4784 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10
[junit4] 2> 1041008 T4784 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10/index.20140608212708547 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10/index.20140608212708547;done=false>>]
[junit4] 2> 1041008 T4784 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001/tempDir-001/jetty10/index.20140608212708547
[junit4] 2> 1041008 T4784 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:243)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:346)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:330)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:190)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:444)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:208)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1041009 T4784 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-22ACEAC452226E54-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=mt, timezone=SystemV/AST4ADT
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_20-ea (64-bit)/cpus=8,threads=1,free=229307912,total=479436800
[junit4] 2> NOTE: All tests run in this JVM: [TestSweetSpotSimilarityFactory, TestLazyCores, TestAddFieldRealTimeGet, TestLRUCache, TestDistributedSearch, TestRecovery, TestSolrDeletionPolicy2, RequestHandlersTest, SSLMigrationTest, TestCloudInspectUtil, ZkNodePropsTest, TermVectorComponentDistributedTest, HdfsCollectionsAPIDistributedZkTest, HdfsRecoveryZkTest, TestMaxScoreQueryParser, HighlighterConfigTest, MinimalSchemaTest, DOMUtilTest, ConvertedLegacyTest, TriLevelCompositeIdRoutingTest, RemoteQueryErrorTest, TestFieldCollectionResource, TestInfoStreamLogging, BasicZkTest, TestFieldTypeCollectionResource, BlockDirectoryTest, TestDFRSimilarityFactory, DirectUpdateHandlerTest, TestDefaultSearchFieldResource, TestWordDelimiterFilterFactory, FullSolrCloudDistribCmdsTest, TestReRankQParserPlugin, TestSolrQueryParserDefaultOperatorResource, ResponseHeaderTest, TestDynamicFieldCollectionResource, TestDocumentBuilder, TestRTGBase, DistanceFunctionTest, TestIBSimilarityFactory, NumericFieldsTest, CachingDirectoryFactoryTest, UniqFieldsUpdateProcessorFactoryTest, TestStressLucene, TestCollationField, TestOmitPositions, StatsComponentTest, CurrencyFieldOpenExchangeTest, LegacyHTMLStripCharFilterTest, TestManagedResource, ParsingFieldUpdateProcessorsTest, TestManagedStopFilterFactory, ConnectionManagerTest, CoreContainerCoreInitFailuresTest, BadCopyFieldTest, DateMathParserTest, AlternateDirectoryTest, SignatureUpdateProcessorFactoryTest, DistribDocExpirationUpdateProcessorTest, TestZkChroot, AnalysisErrorHandlingTest, OpenCloseCoreStressTest, CoreMergeIndexesAdminHandlerTest, SolrXmlInZkTest, BasicDistributedZk2Test, TestManagedSynonymFilterFactory, TestFreeTextSuggestions, DocumentAnalysisRequestHandlerTest, SystemInfoHandlerTest, SolrCmdDistributorTest, TestRecoveryHdfs, TestStressReorder, ShardRoutingCustomTest, TestRangeQuery, SuggesterWFSTTest, TestCollapseQParserPlugin, FieldFacetTest, SpellCheckCollatorTest, FieldFacetExtrasTest, TestReplicationHandlerBackup, TestDocBasedVersionConstraints, InfoHandlerTest, MultiThreadedOCPTest, TestBinaryField, OutputWriterTest, AbstractAnalyticsStatsTest, TestComponentsName, SimplePostToolTest, FieldAnalysisRequestHandlerTest, PingRequestHandlerTest, TestSolrJ, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J1 in 144.28s, 1 test, 1 failure <<< FAILURES!
[...truncated 663 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 410 suites, 1693 tests, 1 failure, 36 ignored (15 assumptions)
Total time: 62 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b15 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any