You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/30 08:20:10 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
6375 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6375/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
2 tests failed.
REGRESSION: org.apache.solr.update.AutoCommitTest.testMaxTime
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([1FAB55745AC80A43:855F2896C452967F]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:532)
at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1218</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1439246718290886656</long><int name="intDefault">42</int><date name="timestamp">2013-06-30T06:10:30.206Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
</response>
request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:525)
... 41 more
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:57191/ag_yu/onenodecollectioncore returned non ok status:404, message:Can not find: /ag_yu/onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:57191/ag_yu/onenodecollectioncore returned non ok status:404, message:Can not find: /ag_yu/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([1FAB55745AC80A43:9E4DDB6C2D976A7F]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Build Log:
[...truncated 9548 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 571260 T5032 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ag_yu/
[junit4:junit4] 2> 571266 T5032 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407
[junit4:junit4] 2> 571267 T5032 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 571268 T5033 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 571368 T5032 oasc.ZkTestServer.run start zk server on port:34977
[junit4:junit4] 2> 571369 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 571878 T5039 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@def53fb0 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 571878 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 571881 T5032 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 571889 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 571890 T5041 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bdcd1ff name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 571890 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 571891 T5032 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 571893 T5032 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 571895 T5032 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 571897 T5032 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 571899 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 571899 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 571903 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 571903 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 571906 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 571907 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 571909 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 571910 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 571912 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 571913 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 571915 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 571916 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 571918 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 571918 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 571921 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 571921 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 571923 T5032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 571924 T5032 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 571989 T5032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 571992 T5032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4] 2> 571993 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 571994 T5032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 571994 T5032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066
[junit4:junit4] 2> 571995 T5032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/solr.xml
[junit4:junit4] 2> 571995 T5032 oasc.CoreContainer.<init> New CoreContainer -279125625
[junit4:junit4] 2> 571996 T5032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/'
[junit4:junit4] 2> 571997 T5032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/'
[junit4:junit4] 2> 572036 T5032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 572037 T5032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 572038 T5032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 572038 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 572039 T5032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 572039 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 572040 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 572041 T5032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 572041 T5032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 572042 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 572046 T5032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 572047 T5032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34977/solr
[junit4:junit4] 2> 572048 T5032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 572048 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 572050 T5052 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2240eb00 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 572050 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 572051 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 572056 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 572057 T5054 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22b7b14e name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 572057 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 572059 T5032 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 572061 T5032 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 572063 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 572065 T5032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39229_ag_yu
[junit4:junit4] 2> 572066 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39229_ag_yu
[junit4:junit4] 2> 572068 T5032 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 572072 T5032 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 572074 T5032 oasc.Overseer.start Overseer (id=89952917743861763-127.0.0.1:39229_ag_yu-n_0000000000) starting
[junit4:junit4] 2> 572078 T5032 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 572082 T5056 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 572082 T5032 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 572084 T5032 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 572086 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 572088 T5055 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 572090 T5057 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 572090 T5057 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 572091 T5057 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 573591 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 573592 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 573592 T5055 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 573592 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 573595 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 574092 T5057 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 574092 T5057 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/collection1
[junit4:junit4] 2> 574093 T5057 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 574094 T5057 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 574095 T5057 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 574096 T5057 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/collection1/'
[junit4:junit4] 2> 574097 T5057 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/collection1/lib/README' to classloader
[junit4:junit4] 2> 574097 T5057 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 574146 T5057 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 574177 T5057 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 574178 T5057 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 574182 T5057 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 574489 T5057 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 574494 T5057 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 574497 T5057 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 574507 T5057 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 574511 T5057 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 574514 T5057 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 574515 T5057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 574516 T5057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 574516 T5057 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 574517 T5057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 574517 T5057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 574518 T5057 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 574518 T5057 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1372572598066/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/
[junit4:junit4] 2> 574519 T5057 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 574519 T5057 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data
[junit4:junit4] 2> 574520 T5057 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index/
[junit4:junit4] 2> 574520 T5057 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 574521 T5057 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index
[junit4:junit4] 2> 574522 T5057 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7ede90ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@488eb5fd),segFN=segments_1,generation=1}
[junit4:junit4] 2> 574522 T5057 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 574524 T5057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 574524 T5057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 574525 T5057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 574525 T5057 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 574526 T5057 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 574526 T5057 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 574526 T5057 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 574527 T5057 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 574527 T5057 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 574528 T5057 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 574530 T5057 oass.SolrIndexSearcher.<init> Opening Searcher@4e5a3ad3 main
[junit4:junit4] 2> 574531 T5057 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 574531 T5057 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 574535 T5058 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e5a3ad3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 574535 T5057 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 574535 T5057 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39229/ag_yu collection:control_collection shard:shard1
[junit4:junit4] 2> 574536 T5057 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 574541 T5057 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 574543 T5057 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 574543 T5057 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 574543 T5057 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39229/ag_yu/collection1/
[junit4:junit4] 2> 574544 T5057 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 574544 T5057 oasc.SyncStrategy.syncToMe http://127.0.0.1:39229/ag_yu/collection1/ has no replicas
[junit4:junit4] 2> 574545 T5057 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39229/ag_yu/collection1/ shard1
[junit4:junit4] 2> 574545 T5057 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 575098 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 575111 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 575150 T5057 oasc.ZkController.register We are http://127.0.0.1:39229/ag_yu/collection1/ and leader is http://127.0.0.1:39229/ag_yu/collection1/
[junit4:junit4] 2> 575151 T5057 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39229/ag_yu
[junit4:junit4] 2> 575152 T5057 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 575152 T5057 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 575153 T5057 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 575154 T5057 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 575156 T5032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 575156 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 575157 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 575161 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 575163 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 575164 T5061 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c0ec69 name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 575164 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 575166 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 575168 T5032 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 575233 T5032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 575235 T5032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4] 2> 575235 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 575236 T5032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 575237 T5032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308
[junit4:junit4] 2> 575237 T5032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/solr.xml
[junit4:junit4] 2> 575238 T5032 oasc.CoreContainer.<init> New CoreContainer 1201807503
[junit4:junit4] 2> 575238 T5032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/'
[junit4:junit4] 2> 575239 T5032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/'
[junit4:junit4] 2> 575297 T5032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 575298 T5032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 575299 T5032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 575300 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 575300 T5032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 575301 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 575301 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 575302 T5032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 575303 T5032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 575303 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 575308 T5032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 575308 T5032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34977/solr
[junit4:junit4] 2> 575309 T5032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 575310 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 575311 T5072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e8cf4ff6 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 575312 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 575313 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 575318 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 575319 T5074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34823ee2 name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 575319 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 575322 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 576325 T5032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37509_ag_yu
[junit4:junit4] 2> 576326 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37509_ag_yu
[junit4:junit4] 2> 576328 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 576328 T5061 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 576329 T5074 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 576330 T5054 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 576333 T5075 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 576333 T5075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 576334 T5075 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 576615 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 576615 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 576618 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 576618 T5055 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 576619 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 576622 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 576622 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 576622 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 577335 T5075 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 577336 T5075 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/collection1
[junit4:junit4] 2> 577336 T5075 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 577337 T5075 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 577338 T5075 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 577339 T5075 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/collection1/'
[junit4:junit4] 2> 577340 T5075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/collection1/lib/README' to classloader
[junit4:junit4] 2> 577340 T5075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 577374 T5075 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 577406 T5075 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 577408 T5075 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 577412 T5075 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 577753 T5075 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 577758 T5075 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 577761 T5075 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 577772 T5075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 577776 T5075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 577780 T5075 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 577781 T5075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 577781 T5075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 577782 T5075 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 577783 T5075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 577783 T5075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 577783 T5075 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 577784 T5075 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/
[junit4:junit4] 2> 577784 T5075 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 577785 T5075 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1
[junit4:junit4] 2> 577785 T5075 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index/
[junit4:junit4] 2> 577786 T5075 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 577786 T5075 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index
[junit4:junit4] 2> 577787 T5075 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@217fa804 lockFactory=org.apache.lucene.store.NativeFSLockFactory@603dc4ad),segFN=segments_1,generation=1}
[junit4:junit4] 2> 577788 T5075 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 577789 T5075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 577790 T5075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 577790 T5075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 577791 T5075 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 577791 T5075 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 577792 T5075 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 577792 T5075 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 577793 T5075 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 577793 T5075 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 577794 T5075 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 577796 T5075 oass.SolrIndexSearcher.<init> Opening Searcher@2a6579e4 main
[junit4:junit4] 2> 577797 T5075 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 577797 T5075 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 577801 T5076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a6579e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 577802 T5075 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 577802 T5075 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37509/ag_yu collection:collection1 shard:shard1
[junit4:junit4] 2> 577803 T5075 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 577811 T5075 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 577813 T5075 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 577814 T5075 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 577814 T5075 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37509/ag_yu/collection1/
[junit4:junit4] 2> 577814 T5075 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 577815 T5075 oasc.SyncStrategy.syncToMe http://127.0.0.1:37509/ag_yu/collection1/ has no replicas
[junit4:junit4] 2> 577815 T5075 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37509/ag_yu/collection1/ shard1
[junit4:junit4] 2> 577815 T5075 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 578127 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 578135 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 578135 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 578135 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 578170 T5075 oasc.ZkController.register We are http://127.0.0.1:37509/ag_yu/collection1/ and leader is http://127.0.0.1:37509/ag_yu/collection1/
[junit4:junit4] 2> 578171 T5075 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37509/ag_yu
[junit4:junit4] 2> 578171 T5075 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 578172 T5075 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 578172 T5075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 578174 T5075 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 578175 T5032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 578176 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 578177 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 578245 T5032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 578247 T5032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4] 2> 578248 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 578248 T5032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 578249 T5032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321
[junit4:junit4] 2> 578250 T5032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/solr.xml
[junit4:junit4] 2> 578250 T5032 oasc.CoreContainer.<init> New CoreContainer -391278647
[junit4:junit4] 2> 578251 T5032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/'
[junit4:junit4] 2> 578251 T5032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/'
[junit4:junit4] 2> 578312 T5032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 578313 T5032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 578314 T5032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 578315 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 578316 T5032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 578318 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 578319 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 578320 T5032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 578321 T5032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 578322 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 578329 T5032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 578331 T5032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34977/solr
[junit4:junit4] 2> 578332 T5032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 578334 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 578335 T5088 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67b00072 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 578335 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 578337 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 578345 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 578346 T5090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9ebe00d7 name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 578346 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 578350 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 579353 T5032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57191_ag_yu
[junit4:junit4] 2> 579355 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57191_ag_yu
[junit4:junit4] 2> 579358 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 579358 T5074 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 579358 T5090 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 579359 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 579359 T5061 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 579360 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 579360 T5054 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 579363 T5091 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 579363 T5091 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 579364 T5091 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 579640 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 579641 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 579644 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 579644 T5055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 579645 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 579647 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 579647 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 579647 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 579647 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 580365 T5091 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 580365 T5091 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/collection1
[junit4:junit4] 2> 580366 T5091 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 580367 T5091 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 580367 T5091 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 580368 T5091 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/collection1/'
[junit4:junit4] 2> 580369 T5091 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/collection1/lib/README' to classloader
[junit4:junit4] 2> 580369 T5091 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 580394 T5091 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 580424 T5091 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 580426 T5091 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 580430 T5091 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 580743 T5091 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 580748 T5091 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 580750 T5091 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 580760 T5091 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 580764 T5091 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 580768 T5091 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 580769 T5091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 580770 T5091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 580770 T5091 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 580771 T5091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 580771 T5091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 580772 T5091 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 580772 T5091 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1372572604321/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/
[junit4:junit4] 2> 580772 T5091 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 580773 T5091 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2
[junit4:junit4] 2> 580773 T5091 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index/
[junit4:junit4] 2> 580774 T5091 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 580774 T5091 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index
[junit4:junit4] 2> 580775 T5091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c76fdb14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@753fa470),segFN=segments_1,generation=1}
[junit4:junit4] 2> 580776 T5091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 580777 T5091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 580778 T5091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 580778 T5091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 580779 T5091 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 580779 T5091 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 580780 T5091 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 580780 T5091 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 580781 T5091 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 580781 T5091 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 580782 T5091 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 580784 T5091 oass.SolrIndexSearcher.<init> Opening Searcher@1d571fc5 main
[junit4:junit4] 2> 580785 T5091 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 580785 T5091 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 580789 T5092 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d571fc5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 580790 T5091 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 580791 T5091 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57191/ag_yu collection:collection1 shard:shard2
[junit4:junit4] 2> 580792 T5091 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 580796 T5091 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 580799 T5091 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 580799 T5091 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 580800 T5091 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57191/ag_yu/collection1/
[junit4:junit4] 2> 580800 T5091 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 580800 T5091 oasc.SyncStrategy.syncToMe http://127.0.0.1:57191/ag_yu/collection1/ has no replicas
[junit4:junit4] 2> 580801 T5091 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57191/ag_yu/collection1/ shard2
[junit4:junit4] 2> 580801 T5091 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 581152 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 581200 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 581200 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 581200 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 581200 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 581205 T5091 oasc.ZkController.register We are http://127.0.0.1:57191/ag_yu/collection1/ and leader is http://127.0.0.1:57191/ag_yu/collection1/
[junit4:junit4] 2> 581206 T5091 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57191/ag_yu
[junit4:junit4] 2> 581206 T5091 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 581207 T5091 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 581207 T5091 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 581208 T5091 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 581209 T5032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 581210 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 581210 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 581282 T5032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 581284 T5032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4] 2> 581285 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 581285 T5032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 581286 T5032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355
[junit4:junit4] 2> 581286 T5032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/solr.xml
[junit4:junit4] 2> 581287 T5032 oasc.CoreContainer.<init> New CoreContainer -1142665165
[junit4:junit4] 2> 581288 T5032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/'
[junit4:junit4] 2> 581288 T5032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/'
[junit4:junit4] 2> 581345 T5032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 581346 T5032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 581346 T5032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 581347 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 581348 T5032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 581349 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 581349 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 581350 T5032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 581351 T5032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 581352 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 581357 T5032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 581359 T5032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34977/solr
[junit4:junit4] 2> 581360 T5032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 581361 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 581362 T5104 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72002e17 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 581362 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 581364 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 581369 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 581370 T5106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@418bf1de name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 581370 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 581373 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 582375 T5032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55392_ag_yu
[junit4:junit4] 2> 582378 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55392_ag_yu
[junit4:junit4] 2> 582381 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 582382 T5061 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 582382 T5074 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 582383 T5090 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 582383 T5106 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 582384 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582385 T5054 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 582385 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582383 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582392 T5107 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 582393 T5107 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 582395 T5107 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 582705 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 582706 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 582708 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 582709 T5055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 582709 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 582711 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582711 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582711 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582711 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 582711 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 583396 T5107 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 583396 T5107 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/collection1
[junit4:junit4] 2> 583397 T5107 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 583397 T5107 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 583398 T5107 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 583399 T5107 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/collection1/'
[junit4:junit4] 2> 583399 T5107 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/collection1/lib/README' to classloader
[junit4:junit4] 2> 583400 T5107 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 583424 T5107 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 583455 T5107 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 583456 T5107 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 583461 T5107 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 583777 T5107 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 583782 T5107 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 583785 T5107 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 583794 T5107 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 583798 T5107 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 583802 T5107 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 583803 T5107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 583804 T5107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 583804 T5107 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 583805 T5107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 583805 T5107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 583806 T5107 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 583806 T5107 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1372572607355/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/
[junit4:junit4] 2> 583807 T5107 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 583808 T5107 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3
[junit4:junit4] 2> 583808 T5107 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index/
[junit4:junit4] 2> 583808 T5107 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 583809 T5107 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index
[junit4:junit4] 2> 583810 T5107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@79d7afa6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30e9f7ab),segFN=segments_1,generation=1}
[junit4:junit4] 2> 583811 T5107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 583812 T5107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 583813 T5107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 583813 T5107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 583814 T5107 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 583814 T5107 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 583815 T5107 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 583815 T5107 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 583816 T5107 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 583816 T5107 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 583818 T5107 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 583820 T5107 oass.SolrIndexSearcher.<init> Opening Searcher@4e9fcb70 main
[junit4:junit4] 2> 583820 T5107 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 583821 T5107 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 583825 T5108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e9fcb70 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 583826 T5107 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 583826 T5107 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55392/ag_yu collection:collection1 shard:shard1
[junit4:junit4] 2> 583829 T5107 oasc.ZkController.register We are http://127.0.0.1:55392/ag_yu/collection1/ and leader is http://127.0.0.1:37509/ag_yu/collection1/
[junit4:junit4] 2> 583830 T5107 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55392/ag_yu
[junit4:junit4] 2> 583830 T5107 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 583830 T5107 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C632 name=collection1 org.apache.solr.core.SolrCore@849f18e8 url=http://127.0.0.1:55392/ag_yu/collection1 node=127.0.0.1:55392_ag_yu C632_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55392/ag_yu, state=down, node_name=127.0.0.1:55392_ag_yu, core=collection1}
[junit4:junit4] 2> 583831 T5109 C632 P55392 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 583831 T5107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 583831 T5109 C632 P55392 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 583832 T5109 C632 P55392 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 583832 T5032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 583834 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 583833 T5109 C632 P55392 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 583834 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 583835 T5109 C632 P55392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 583844 T5070 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 583907 T5032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 583909 T5032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4] 2> 583910 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 583910 T5032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 583911 T5032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979
[junit4:junit4] 2> 583911 T5032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/solr.xml
[junit4:junit4] 2> 583912 T5032 oasc.CoreContainer.<init> New CoreContainer -1095339754
[junit4:junit4] 2> 583913 T5032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/'
[junit4:junit4] 2> 583913 T5032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/'
[junit4:junit4] 2> 583972 T5032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 583973 T5032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 583973 T5032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 583974 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 583975 T5032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 583975 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 583976 T5032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 583976 T5032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 583977 T5032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 583978 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 583982 T5032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 583983 T5032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34977/solr
[junit4:junit4] 2> 583984 T5032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 583985 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 583986 T5121 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ec3aadaf name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 583986 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 583988 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 583993 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 583994 T5123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ba41c251 name:ZooKeeperConnection Watcher:127.0.0.1:34977/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 583994 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 583997 T5032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 584217 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 584219 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 584225 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584226 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584226 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584226 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584226 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584226 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 584844 T5070 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 584846 T5070 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node3&wt=javabin&nodeName=127.0.0.1:55392_ag_yu&core=collection1} status=0 QTime=1002
[junit4:junit4] 2> 585000 T5032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40889_ag_yu
[junit4:junit4] 2> 585002 T5032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40889_ag_yu
[junit4:junit4] 2> 585006 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 585009 T5074 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585009 T5123 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585009 T5090 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585010 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585010 T5106 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585011 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585009 T5061 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585013 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585010 T5054 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 585011 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585015 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585018 T5124 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 585019 T5124 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 585020 T5124 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 585734 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 585735 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40889/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "node_name":"127.0.0.1:40889_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 585736 T5055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 585737 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 585742 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585743 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585743 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585743 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585743 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 585743 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 586021 T5124 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 586022 T5124 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/collection1
[junit4:junit4] 2> 586023 T5124 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 586025 T5124 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 586026 T5124 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 586028 T5124 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/collection1/'
[junit4:junit4] 2> 586030 T5124 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/collection1/lib/README' to classloader
[junit4:junit4] 2> 586031 T5124 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 586062 T5124 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 586095 T5124 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 586096 T5124 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 586100 T5124 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 586418 T5124 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 586423 T5124 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 586426 T5124 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 586442 T5124 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 586450 T5124 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 586457 T5124 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 586459 T5124 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 586459 T5124 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 586460 T5124 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 586462 T5124 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 586462 T5124 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 586463 T5124 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 586463 T5124 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1372572609979/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/
[junit4:junit4] 2> 586463 T5124 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 586464 T5124 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4
[junit4:junit4] 2> 586465 T5124 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index/
[junit4:junit4] 2> 586465 T5124 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 586465 T5124 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index
[junit4:junit4] 2> 586467 T5124 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8b9bac7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d001bc2e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 586467 T5124 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 586469 T5124 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 586469 T5124 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 586470 T5124 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 586471 T5124 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 586472 T5124 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 586472 T5124 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 586473 T5124 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 586474 T5124 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 586475 T5124 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 586477 T5124 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 586480 T5124 oass.SolrIndexSearcher.<init> Opening Searcher@44401400 main
[junit4:junit4] 2> 586481 T5124 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 586481 T5124 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 586486 T5125 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44401400 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 586488 T5124 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 586489 T5124 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40889/ag_yu collection:collection1 shard:shard2
[junit4:junit4] 2> 586492 T5124 oasc.ZkController.register We are http://127.0.0.1:40889/ag_yu/collection1/ and leader is http://127.0.0.1:57191/ag_yu/collection1/
[junit4:junit4] 2> 586492 T5124 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40889/ag_yu
[junit4:junit4] 2> 586493 T5124 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 586493 T5124 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C633 name=collection1 org.apache.solr.core.SolrCore@c072e632 url=http://127.0.0.1:40889/ag_yu/collection1 node=127.0.0.1:40889_ag_yu C633_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:40889/ag_yu, state=down, node_name=127.0.0.1:40889_ag_yu, core=collection1}
[junit4:junit4] 2> 586494 T5126 C633 P40889 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 586494 T5126 C633 P40889 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 586494 T5124 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 586495 T5126 C633 P40889 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 586495 T5126 C633 P40889 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 586496 T5032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 586497 T5032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 586496 T5126 C633 P40889 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 586497 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 586502 T5032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 586502 T5083 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 586503 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 586509 T5070 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 586510 T5070 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> ASYNC NEW_CORE C634 name=collection1 org.apache.solr.core.SolrCore@849f18e8 url=http://127.0.0.1:55392/ag_yu/collection1 node=127.0.0.1:55392_ag_yu C634_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55392/ag_yu, state=recovering, node_name=127.0.0.1:55392_ag_yu, core=collection1}
[junit4:junit4] 2> 586848 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37509/ag_yu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 586848 T5109 C634 P55392 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55392/ag_yu START replicas=[http://127.0.0.1:37509/ag_yu/collection1/] nUpdates=100
[junit4:junit4] 2> 586849 T5109 C634 P55392 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 586849 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 586850 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 586850 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 586851 T5109 C634 P55392 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 586851 T5109 C634 P55392 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37509/ag_yu/collection1/. core=collection1
[junit4:junit4] 2> 586851 T5109 C634 P55392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C635 name=collection1 org.apache.solr.core.SolrCore@1f4c0ce0 url=http://127.0.0.1:37509/ag_yu/collection1 node=127.0.0.1:37509_ag_yu C635_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:37509/ag_yu, state=active, node_name=127.0.0.1:37509_ag_yu, core=collection1, leader=true}
[junit4:junit4] 2> 586853 T5069 C635 P37509 oasc.SolrCore.execute [collection1] webapp=/ag_yu path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 586858 T5069 C635 P37509 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 586860 T5069 C635 P37509 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@217fa804 lockFactory=org.apache.lucene.store.NativeFSLockFactory@603dc4ad),segFN=segments_1,generation=1}
[junit4:junit4] 2> 586860 T5069 C635 P37509 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 586861 T5069 C635 P37509 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 586862 T5069 C635 P37509 oass.SolrIndexSearcher.<init> Opening Searcher@1d338f2f realtime
[junit4:junit4] 2> 586862 T5069 C635 P37509 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 586863 T5069 C635 P37509 oasup.LogUpdateProcessor.finish [collection1] webapp=/ag_yu path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 5
[junit4:junit4] 2> 586863 T5109 C634 P55392 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 586864 T5109 C634 P55392 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 586866 T5067 C635 P37509 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 586866 T5067 C635 P37509 oasc.SolrCore.execute [collection1] webapp=/ag_yu path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 586867 T5109 C634 P55392 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 586867 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 586868 T5109 C634 P55392 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 586868 T5109 C634 P55392 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 586869 T5109 C634 P55392 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 587248 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 587249 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40889/ag_yu",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "node_name":"127.0.0.1:40889_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 587252 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 587252 T5055 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 587253 T5055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 587255 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 587259 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587259 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587259 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587260 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587259 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587259 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 587504 T5083 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 587505 T5083 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node4&wt=javabin&nodeName=127.0.0.1:40889_ag_yu&core=collection1} status=0 QTime=1003
[junit4:junit4] 2> 587511 T5070 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for onenodecollectioncore
[junit4:junit4] 2> 587511 T5070 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/onenodecollectioncore
[junit4:junit4] 2> 587512 T5070 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 587513 T5070 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 587514 T5070 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 587515 T5070 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 587515 T5070 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 587517 T5070 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 587519 T5070 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/onenodecollectioncore/'
[junit4:junit4] 2> 587544 T5070 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 587588 T5070 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 587590 T5070 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 587594 T5070 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4:junit4] 2> 587913 T5070 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 587918 T5070 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 587921 T5070 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 587931 T5070 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 587937 T5070 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 587943 T5070 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 587945 T5070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 587946 T5070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 587947 T5070 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 587948 T5070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 587949 T5070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 587950 T5070 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 587951 T5070 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1372572601308/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/
[junit4:junit4] 2> 587952 T5070 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fab4487f
[junit4:junit4] 2> 587954 T5070 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection
[junit4:junit4] 2> 587955 T5070 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index/
[junit4:junit4] 2> 587956 T5070 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 587957 T5070 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index
[junit4:junit4] 2> 587959 T5070 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c297484 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a5dd44b2),segFN=segments_1,generation=1}
[junit4:junit4] 2> 587960 T5070 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 587961 T5070 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 587962 T5070 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 587963 T5070 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 587963 T5070 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 587964 T5070 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 587965 T5070 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 587965 T5070 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 587966 T5070 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 587967 T5070 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 587968 T5070 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 587970 T5070 oass.SolrIndexSearcher.<init> Opening Searcher@fae01f94 main
[junit4:junit4] 2> 587972 T5070 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 587972 T5070 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 587975 T5129 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@fae01f94 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 587977 T5070 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 587977 T5070 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:37509/ag_yu collection:onenodecollection shard:shard1
[junit4:junit4] 2> 587978 T5070 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 587983 T5070 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 587985 T5070 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 587985 T5070 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 587986 T5070 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37509/ag_yu/onenodecollectioncore/
[junit4:junit4] 2> 587986 T5070 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 587987 T5070 oasc.SyncStrategy.syncToMe http://127.0.0.1:37509/ag_yu/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 587988 T5070 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37509/ag_yu/onenodecollectioncore/ shard1
[junit4:junit4] 2> 587988 T5070 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 588767 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 588781 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588781 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588781 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588781 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588782 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588781 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 588794 T5070 oasc.ZkController.register We are http://127.0.0.1:37509/ag_yu/onenodecollectioncore/ and leader is http://127.0.0.1:37509/ag_yu/onenodecollectioncore/
[junit4:junit4] 2> 588796 T5070 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:37509/ag_yu
[junit4:junit4] 2> 588797 T5070 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 588799 T5070 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 588803 T5070 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 588806 T5070 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=2297
[junit4:junit4] 2> 588808 T5032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 588810 T5032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C636 name=collection1 org.apache.solr.core.SolrCore@c072e632 url=http://127.0.0.1:40889/ag_yu/collection1 node=127.0.0.1:40889_ag_yu C636_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:40889/ag_yu, state=recovering, node_name=127.0.0.1:40889_ag_yu, core=collection1}
[junit4:junit4] 2> 589505 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57191/ag_yu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 589506 T5126 C636 P40889 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40889/ag_yu START replicas=[http://127.0.0.1:57191/ag_yu/collection1/] nUpdates=100
[junit4:junit4] 2> 589506 T5126 C636 P40889 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 589507 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 589507 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 589507 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 589507 T5126 C636 P40889 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 589508 T5126 C636 P40889 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57191/ag_yu/collection1/. core=collection1
[junit4:junit4] 2> 589508 T5126 C636 P40889 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C637 name=collection1 org.apache.solr.core.SolrCore@ff4e49ec url=http://127.0.0.1:57191/ag_yu/collection1 node=127.0.0.1:57191_ag_yu C637_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57191/ag_yu, state=active, node_name=127.0.0.1:57191_ag_yu, core=collection1, leader=true}
[junit4:junit4] 2> 589513 T5085 C637 P57191 oasc.SolrCore.execute [collection1] webapp=/ag_yu path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 589516 T5083 C637 P57191 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 589517 T5083 C637 P57191 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c76fdb14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@753fa470),segFN=segments_1,generation=1}
[junit4:junit4] 2> 589518 T5083 C637 P57191 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 589518 T5083 C637 P57191 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 589519 T5083 C637 P57191 oass.SolrIndexSearcher.<init> Opening Searcher@d23d76d6 realtime
[junit4:junit4] 2> 589520 T5083 C637 P57191 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 589520 T5083 C637 P57191 oasup.LogUpdateProcessor.finish [collection1] webapp=/ag_yu path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 4
[junit4:junit4] 2> 589521 T5126 C636 P40889 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 589522 T5126 C636 P40889 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 589523 T5086 C637 P57191 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 589524 T5086 C637 P57191 oasc.SolrCore.execute [collection1] webapp=/ag_yu path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 589525 T5126 C636 P40889 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 589525 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 589525 T5126 C636 P40889 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 589525 T5126 C636 P40889 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 589526 T5126 C636 P40889 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 589813 T5032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 590293 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 590294 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 590298 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40889/ag_yu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "node_name":"127.0.0.1:40889_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 590301 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590302 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590302 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590302 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590302 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590302 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 590816 T5032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 590817 T5032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 590819 T5032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 590825 T5032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 590826 T5132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9f107e03 name:ZooKeeperConnection Watcher:127.0.0.1:34977 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 590826 T5032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 590862 T5032 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 39229
[junit4:junit4] 2> 590862 T5032 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-279125625
[junit4:junit4] 2> 591807 T5055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 591809 T5055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 591810 T5055 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 591816 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591816 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591816 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591816 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591816 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 592864 T5032 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 592865 T5032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 592867 T5032 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@b42a2c50
[junit4:junit4] 2> 592870 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 592871 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 592871 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 592872 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 592872 T5032 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 592873 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 592873 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index;done=false>>]
[junit4:junit4] 2> 592874 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data/index
[junit4:junit4] 2> 592874 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data;done=false>>]
[junit4:junit4] 2> 592875 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/control/data
[junit4:junit4] 2> 592876 T5055 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89952917743861763-127.0.0.1:39229_ag_yu-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 592876 T5054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 592877 T5123 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 592877 T5074 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 592877 T5106 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 592877 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592878 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592877 T5090 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 592879 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592880 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592882 T5074 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 592884 T5074 oasc.Overseer.start Overseer (id=89952917743861766-127.0.0.1:37509_ag_yu-n_0000000001) starting
[junit4:junit4] 2> 592886 T5134 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 592887 T5133 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 592888 T5133 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 592889 T5133 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 592891 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592891 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592891 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592891 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 592898 T5032 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ag_yu,null}
[junit4:junit4] 2> 592949 T5032 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 37509
[junit4:junit4] 2> 592950 T5032 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1201807503
[junit4:junit4] 2> 594177 T5054 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 594177 T5054 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 594394 T5133 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 594396 T5133 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 594396 T5133 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 594399 T5133 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 594400 T5133 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 594403 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 594403 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 594403 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 594403 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 595953 T5032 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 595954 T5032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 595957 T5032 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 595960 T5032 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1f4c0ce0
[junit4:junit4] 2> 595963 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 595964 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 595964 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 595965 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 595966 T5032 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 595966 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 595967 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1;done=false>>]
[junit4:junit4] 2> 595967 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1
[junit4:junit4] 2> 595968 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index;done=false>>]
[junit4:junit4] 2> 595968 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty1/index
[junit4:junit4] 2> 595969 T5032 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@fe52080b
[junit4:junit4] 2> 595973 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 595974 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 595975 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 595976 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 595977 T5032 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 595977 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 595978 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index;done=false>>]
[junit4:junit4] 2> 595979 T5032 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection/index
[junit4:junit4] 2> 595980 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection;done=false>>]
[junit4:junit4] 2> 595981 T5032 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1372572597407/onenodecollection
[junit4:junit4] 2> 595983 T5133 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89952917743861766-127.0.0.1:37509_ag_yu-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 595984 T5090 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 595984 T5123 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 595984 T5106 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 595984 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 595985 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 595984 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 595987 T5106 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 595987 T5090 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 595988 T5106 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 595989 T5106 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 595989 T5106 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 595989 T5090 oasc.Overseer.start Overseer (id=89952917743861768-127.0.0.1:57191_ag_yu-n_0000000002) starting
[junit4:junit4] 2> 595989 T5106 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55392/ag_yu/collection1/
[junit4:junit4] 2> 595990 T5106 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 595990 T5106 oasc.SyncStrategy.syncToMe http://127.0.0.1:55392/ag_yu/collection1/ has no replicas
[junit4:junit4] 2> 595991 T5106 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55392/ag_yu/collection1/ shard1
[junit4:junit4] 2> 595991 T5106 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 595993 T5136 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 595993 T5135 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 595995 T5135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 595995 T5135 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 595997 T5135 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 596003 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 596003 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 596004 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 596004 T5032 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ag_yu,null}
[junit4:junit4] 2> 596056 T5032 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57191
[junit4:junit4] 2> 596057 T5032 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-391278647
[junit4:junit4] 2> 597059 T5032 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 597061 T5032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 597064 T5032 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@ff4e49ec
[junit4:junit4] 2> 597069 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 597070 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 597071 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 597073 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 597074 T5032 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 597076 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 597077 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index;done=false>>]
[junit4:junit4] 2> 597078 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2/index
[junit4:junit4] 2> 597079 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2;done=false>>]
[junit4:junit4] 2> 597080 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty2
[junit4:junit4] 2> 597082 T5135 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89952917743861768-127.0.0.1:57191_ag_yu-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 597084 T5123 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 597085 T5106 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 597087 T5123 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 597088 T5123 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 597089 T5123 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 597089 T5106 oasc.Overseer.start Overseer (id=89952917743861770-127.0.0.1:55392_ag_yu-n_0000000003) starting
[junit4:junit4] 2> 597090 T5123 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40889/ag_yu/collection1/
[junit4:junit4] 2> 597091 T5123 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40889/ag_yu START replicas=[http://127.0.0.1:57191/ag_yu/collection1/] nUpdates=100
[junit4:junit4] 2> 597092 T5123 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40889/ag_yu DONE. We have no versions. sync failed.
[junit4:junit4] 2> 597093 T5123 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
[junit4:junit4] 2> 597094 T5123 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 597095 T5123 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40889/ag_yu/collection1/ shard2
[junit4:junit4] 2> 597096 T5123 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 597097 T5139 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 597098 T5106 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 597099 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 597100 T5138 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 597104 T5138 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 597104 T5032 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ag_yu,null}
[junit4:junit4] 2> 597106 T5138 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 597106 T5123 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 597107 T5138 oasc.Overseer$ClusterStateUpdater.updateState shard=shard2 is already registered
[junit4:junit4] 2> 597107 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 597112 T5138 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 597125 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 597125 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 597158 T5032 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 55392
[junit4:junit4] 2> 597159 T5032 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1142665165
[junit4:junit4] 2> 597283 T5074 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 597284 T5074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 597284 T5074 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 598162 T5032 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 598162 T5032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 598164 T5032 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@849f18e8
[junit4:junit4] 2> 598167 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 598167 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 598168 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 598168 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 598169 T5032 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 598170 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 598170 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3;done=false>>]
[junit4:junit4] 2> 598171 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3
[junit4:junit4] 2> 598171 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index;done=false>>]
[junit4:junit4] 2> 598172 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty3/index
[junit4:junit4] 2> 598173 T5138 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89952917743861770-127.0.0.1:55392_ag_yu-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 598174 T5123 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 598174 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 598176 T5123 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 598178 T5123 oasc.Overseer.start Overseer (id=89952917743861772-127.0.0.1:40889_ag_yu-n_0000000004) starting
[junit4:junit4] 2> 598181 T5141 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 598181 T5140 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 598183 T5140 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 598183 T5140 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 598184 T5140 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 598186 T5140 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 598189 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 598194 T5032 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ag_yu,null}
[junit4:junit4] 2> 598249 T5032 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 40889
[junit4:junit4] 2> 598250 T5032 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1095339754
[junit4:junit4] 2> 598383 T5090 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 598383 T5090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 598384 T5090 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 599252 T5032 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 599253 T5032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 599255 T5032 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@c072e632
[junit4:junit4] 2> 599265 T5032 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 599266 T5032 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 599267 T5032 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 599268 T5032 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 599270 T5032 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 599271 T5032 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 599272 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index;done=false>>]
[junit4:junit4] 2> 599273 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4/index
[junit4:junit4] 2> 599274 T5032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4;done=false>>]
[junit4:junit4] 2> 599275 T5032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1372572597407/jetty4
[junit4:junit4] 2> 599277 T5140 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89952917743861772-127.0.0.1:40889_ag_yu-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 599299 T5032 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ag_yu,null}
[junit4:junit4] 2> 599359 T5032 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 599361 T5032 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34977 34977
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> / (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr (7)
[junit4:junit4] 1> /solr/configs (1)
[junit4:junit4] 1> /solr/configs/conf1 (9)
[junit4:junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89952917743861763-127.0.0.1:39229_ag_yu-n_0000000000"}
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89952917743861770-127.0.0.1:55392_ag_yu-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89952917743861766-127.0.0.1:37509_ag_yu-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89952917743861763-127.0.0.1:39229_ag_yu-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89952917743861768-127.0.0.1:57191_ag_yu-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89952917743861772-127.0.0.1:40889_ag_yu-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections (3)
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89952917743861763-core_node1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 1> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89952917743861770-core_node3-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89952917743861766-core_node1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89952917743861772-core_node4-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89952917743861768-core_node2-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (2)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 1> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 1> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/onenodecollection (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "router":"compositeId",
[junit4:junit4] 1> "configName":"conf1"}
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election/89952917743861766-core_node1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/onenodecollection/leaders (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 1> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 1> "core":"onenodecollectioncore"}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:57191_ag_yu (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37509_ag_yu (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:39229_ag_yu (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:55392_ag_yu (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:40889_ag_yu (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"core_node1":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39229/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:39229_ag_yu",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "onenodecollection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"core_node1":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 1> "roles":"none",
[junit4:junit4] 1> "core":"onenodecollectioncore",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "shards":{
[junit4:junit4] 1> "shard1":{
[junit4:junit4] 1> "range":"80000000-ffffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "core_node1":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37509/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:37509_ag_yu",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "core_node3":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:55392/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:55392_ag_yu",
[junit4:junit4] 1> "core":"collection1"}}},
[junit4:junit4] 1> "shard2":{
[junit4:junit4] 1> "range":"0-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "core_node2":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57191/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:57191_ag_yu",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "core_node4":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40889/ag_yu",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:40889_ag_yu",
[junit4:junit4] 1> "core":"collection1"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=1FAB55745AC80A43 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=th_TH_TH_#u-nu-thai -Dtests.timezone=Mideast/Riyadh87 -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 28.1s J1 | BasicDistributedZk2Test.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:57191/ag_yu/onenodecollectioncore returned non ok status:404, message:Can not find: /ag_yu/onenodecollectioncore/update
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([1FAB55745AC80A43:9E4DDB6C2D976A7F]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:780)
[junit4:junit4] 2> 599387 T5032 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 28137 T5031 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 599474 T5106 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 599474 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 599475 T5106 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 600578 T5123 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 600579 T5123 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 600579 T5123 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=th_TH_TH_#u-nu-thai, timezone=Mideast/Riyadh87
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=2,free=20101064,total=343736320
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestDefaultSearchFieldResource, TestHashPartitioner, TestQuerySenderNoQuery, SpellCheckCollatorTest, TestArbitra
[...truncated too long message...]
11,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 610979 T1951 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 610981 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[0 (1439246725597364224)]} 0 1
[junit4:junit4] 2> 610981 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[1 (1439246725598412800)]} 0 1
[junit4:junit4] 2> 610982 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1439246725599461376)]} 0 1
[junit4:junit4] 2> 610983 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[3 (1439246725599461377)]} 0 1
[junit4:junit4] 2> 610983 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[4 (1439246725600509952)]} 0 1
[junit4:junit4] 2> 610984 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[5 (1439246725601558528)]} 0 1
[junit4:junit4] 2> 610985 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[6 (1439246725601558529)]} 0 1
[junit4:junit4] 2> 610985 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[7 (1439246725602607104)]} 0 1
[junit4:junit4] 2> 610986 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[8 (1439246725603655680)]} 0 1
[junit4:junit4] 2> 610988 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[9 (1439246725603655681)]} 0 1
[junit4:junit4] 2> 610989 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[10 (1439246725606801408)]} 0 1
[junit4:junit4] 2> 610990 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[11 (1439246725606801409)]} 0 1
[junit4:junit4] 2> 610990 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[12 (1439246725607849984)]} 0 1
[junit4:junit4] 2> 610991 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[13 (1439246725608898560)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C546 name=collection1 org.apache.solr.core.SolrCore@63259c22
[junit4:junit4] 2> 610992 T1951 C546 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:1&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0
[junit4:junit4] 2> 610993 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[14 (1439246725610995712)]} 0 1
[junit4:junit4] 2> 610994 T1958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4] 2> 610997 T1958 oass.SolrIndexSearcher.<init> Opening Searcher@43865a64 main
[junit4:junit4] 2> 610998 T1958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 610999 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43865a64 main{StandardDirectoryReader(segments_3:21:nrt _4(5.0):c10 _5(5.0):c5)}
[junit4:junit4] 2> 611244 T1951 oasup.LogUpdateProcessor.finish [collection1] {add=[15 (1439246725874188288)]} 0 1
[junit4:junit4] 2> 611245 T1951 C546 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:14&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0
[junit4:junit4] 2> 611247 T1951 C546 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:15&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0
[junit4:junit4] 2> 611249 T1951 oas.SolrTestCaseJ4.tearDown ###Ending testMaxDocs
[junit4:junit4] 2> 611251 T1951 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 611251 T1951 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1240616512
[junit4:junit4] 2> 611251 T1951 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@63259c22
[junit4:junit4] 2> 611259 T1951 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommit maxDocs=14,soft autocommits=1,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=16,adds=1,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=16,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 611260 T1951 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 611260 T1951 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 611261 T1951 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 611263 T1951 C546 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dabf87d4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@94c8cf6a)),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dabf87d4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@94c8cf6a)),segFN=segments_4,generation=4}
[junit4:junit4] 2> 611263 T1951 C546 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 611264 T1951 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 611265 T1951 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 611265 T1951 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102/index;done=false>>]
[junit4:junit4] 2> 611266 T1951 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102/index
[junit4:junit4] 2> 611266 T1951 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102;done=false>>]
[junit4:junit4] 2> 611266 T1951 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372572628102
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_si=PostingsFormat(name=NestedPulsing), range_facet_l=Lucene41(blocksize=128), intDefault=Lucene41(blocksize=128), range_facet_sl=PostingsFormat(name=Asserting), timestamp=Lucene41(blocksize=128), _version_=Lucene41(blocksize=128), multiDefault=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Memory doPackFST= false), subject=PostingsFormat(name=NestedPulsing), text=PostingsFormat(name=Asserting), field_t=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=uk_UA, timezone=Etc/GMT+10
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=57817080,total=184614912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [OverseerTest, QueryElevationComponentTest, TestCoreContainer, TestValueSourceCache, AlternateDirectoryTest, ExternalFileFieldSortTest, TestSolrDeletionPolicy1, AssignTest, MinimalSchemaTest, UUIDFieldTest, TestPhraseSuggestions, CSVRequestHandlerTest, TermsComponentTest, SystemInfoHandlerTest, TestPHPSerializedResponseWriter, FastVectorHighlighterTest, DateFieldTest, SpatialFilterTest, TermVectorComponentTest, SliceStateTest, ResourceLoaderTest, HdfsCollectionsAPIDistributedZkTest, ShowFileRequestHandlerTest, TestDocSet, ShardRoutingTest, MoreLikeThisHandlerTest, OverseerCollectionProcessorTest, TestSchemaNameResource, TestPluginEnable, PolyFieldTest, SOLR749Test, TestQuerySenderListener, SchemaVersionSpecificBehaviorTest, TestStressVersions, DocumentBuilderTest, LukeRequestHandlerTest, ShardSplitTest, DocValuesTest, BasicFunctionalityTest, TestFuzzyAnalyzedSuggestions, TestManagedSchema, TestRealTimeGet, FieldMutatingUpdateProcessorTest, ScriptEngineTest, BlockCacheTest, BadComponentTest, TestSort, TestJoin, DirectUpdateHandlerOptimizeTest, PluginInfoTest, DistributedQueryElevationComponentTest, QueryEqualityTest, TestSolrQueryParserDefaultOperatorResource, SolrCoreTest, TestFieldCollectionResource, SyncSliceTest, HdfsUnloadDistributedZkTest, LoggingHandlerTest, TestFaceting, ClusterStateUpdateTest, AutoCommitTest]
[junit4:junit4] Completed on J0 in 9.39s, 3 tests, 1 error <<< FAILURES!
[...truncated 537 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:379: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:190: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:443: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1245: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 errors, 21 ignored (14 assumptions)
Total time: 40 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure