You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/07/18 16:57:03 UTC

[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk7) - Build # 6544 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6544/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

2 tests failed.
REGRESSION:  org.apache.solr.update.AutoCommitTest.testMaxTime

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([293A2C1EE6652D0E:B3CE51FC78FFB132]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:609)
	at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:227)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">3856</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1440910322474942464</long><int name="intDefault">42</int><date name="timestamp">2013-07-18T14:52:46.789Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
</response>

	request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:602)
	... 41 more


FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Server at http://127.0.0.1:43257/_sgt/uq/onenodecollectioncore returned non ok status:404, message:Can not find: /_sgt/uq/onenodecollectioncore/update

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:43257/_sgt/uq/onenodecollectioncore returned non ok status:404, message:Can not find: /_sgt/uq/onenodecollectioncore/update
	at __randomizedtesting.SeedInfo.seed([293A2C1EE6652D0E:A8DCA206913A4D32]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)




Build Log:
[...truncated 9667 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
   [junit4]   2> 417644 T1578 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_sgt/uq
   [junit4]   2> 417648 T1578 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1374158437984
   [junit4]   2> 417649 T1578 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 417649 T1579 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 417749 T1578 oasc.ZkTestServer.run start zk server on port:41008
   [junit4]   2> 417752 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 417756 T1585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44cbb660 name:ZooKeeperConnection Watcher:127.0.0.1:41008 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 417757 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 417758 T1578 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 417765 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 417767 T1587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a87de355 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 417768 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 417768 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 417773 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 417775 T1578 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 417778 T1578 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 417782 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 417783 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 417798 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 417799 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 417809 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 417811 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 417817 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 417818 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 417824 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 417825 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 417828 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 417829 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 417832 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 417832 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 417835 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 417836 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 417841 T1578 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 417842 T1578 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 417974 T1578 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 417980 T1578 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38359
   [junit4]   2> 417981 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 417981 T1578 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 417982 T1578 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183
   [junit4]   2> 417983 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/'
   [junit4]   2> 417990 T1578 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/solr.xml
   [junit4]   2> 418021 T1578 oasc.CoreContainer.<init> New CoreContainer 1011160096
   [junit4]   2> 418022 T1578 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/]
   [junit4]   2> 418023 T1578 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 418024 T1578 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 418024 T1578 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 418025 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 418026 T1578 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 418026 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 418027 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 418027 T1578 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 418028 T1578 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 418029 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 418033 T1578 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 418034 T1578 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41008/solr
   [junit4]   2> 418034 T1578 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 418036 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 418038 T1598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc82e96 name:ZooKeeperConnection Watcher:127.0.0.1:41008 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 418038 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 418039 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 418046 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 418048 T1600 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@220719d4 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 418048 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 418049 T1578 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 418052 T1578 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 418055 T1578 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 418056 T1578 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38359__sgt%2Fuq
   [junit4]   2> 418057 T1578 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38359__sgt%2Fuq
   [junit4]   2> 418061 T1578 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 418065 T1578 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 418067 T1578 oasc.Overseer.start Overseer (id=90056847391981571-127.0.0.1:38359__sgt%2Fuq-n_0000000000) starting
   [junit4]   2> 418070 T1578 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 418073 T1602 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 418074 T1578 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 418076 T1578 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 418078 T1578 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 418080 T1601 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 418084 T1603 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 418084 T1603 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 418085 T1603 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 419583 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 419584 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:38359/_sgt/uq",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:38359__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 419584 T1601 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
   [junit4]   2> 419585 T1601 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 419587 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 420085 T1603 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 420086 T1603 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/collection1
   [junit4]   2> 420086 T1603 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 420087 T1603 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 420087 T1603 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 420088 T1603 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/collection1/'
   [junit4]   2> 420089 T1603 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/collection1/lib/README' to classloader
   [junit4]   2> 420089 T1603 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/collection1/lib/classes/' to classloader
   [junit4]   2> 420132 T1603 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 420167 T1603 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 420169 T1603 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 420175 T1603 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 420533 T1603 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 420534 T1603 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 420535 T1603 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 420540 T1603 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 420543 T1603 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 420553 T1603 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 420558 T1603 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 420562 T1603 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 420563 T1603 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 420563 T1603 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 420564 T1603 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 420565 T1603 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 420565 T1603 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 420565 T1603 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 420566 T1603 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1374158438183/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/control/data/
   [junit4]   2> 420566 T1603 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d90f5ce9
   [junit4]   2> 420567 T1603 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/control/data
   [junit4]   2> 420568 T1603 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/control/data/index/
   [junit4]   2> 420568 T1603 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 420568 T1603 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/control/data/index
   [junit4]   2> 420570 T1603 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@29ebcb72 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e8834d01)),segFN=segments_1,generation=1}
   [junit4]   2> 420570 T1603 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 420572 T1603 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 420573 T1603 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 420573 T1603 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 420574 T1603 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 420574 T1603 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 420575 T1603 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 420575 T1603 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 420576 T1603 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 420576 T1603 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 420577 T1603 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 420579 T1603 oass.SolrIndexSearcher.<init> Opening Searcher@d1c30ed0 main
   [junit4]   2> 420580 T1603 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 420581 T1603 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 420583 T1604 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d1c30ed0 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 420584 T1603 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 420584 T1603 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38359/_sgt/uq collection:control_collection shard:shard1
   [junit4]   2> 420584 T1603 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
   [junit4]   2> 420589 T1603 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 420594 T1603 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 420597 T1603 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 420597 T1603 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 420597 T1603 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38359/_sgt/uq/collection1/
   [junit4]   2> 420598 T1603 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 420598 T1603 oasc.SyncStrategy.syncToMe http://127.0.0.1:38359/_sgt/uq/collection1/ has no replicas
   [junit4]   2> 420599 T1603 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38359/_sgt/uq/collection1/ shard1
   [junit4]   2> 420599 T1603 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 421091 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 421097 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 421105 T1603 oasc.ZkController.register We are http://127.0.0.1:38359/_sgt/uq/collection1/ and leader is http://127.0.0.1:38359/_sgt/uq/collection1/
   [junit4]   2> 421105 T1603 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38359/_sgt/uq
   [junit4]   2> 421106 T1603 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 421106 T1603 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 421106 T1603 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 421108 T1603 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 421109 T1578 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 421110 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 421110 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 421115 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:
   [junit4]   2> 421117 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 421119 T1607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e76da613 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421119 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 421120 T1578 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 421122 T1578 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
   [junit4]   2> 421229 T1578 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 421235 T1578 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52685
   [junit4]   2> 421236 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 421237 T1578 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 421238 T1578 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458
   [junit4]   2> 421239 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/'
   [junit4]   2> 421248 T1578 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/solr.xml
   [junit4]   2> 421286 T1578 oasc.CoreContainer.<init> New CoreContainer 427635340
   [junit4]   2> 421287 T1578 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/]
   [junit4]   2> 421289 T1578 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 421290 T1578 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 421291 T1578 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 421291 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 421292 T1578 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 421293 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 421294 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 421295 T1578 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 421295 T1578 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 421296 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 421301 T1578 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 421302 T1578 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41008/solr
   [junit4]   2> 421303 T1578 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 421306 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 421308 T1618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fa6430b5 name:ZooKeeperConnection Watcher:127.0.0.1:41008 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421308 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 421310 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 421317 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 421319 T1620 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42d3e71 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421319 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 421322 T1578 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 422325 T1578 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52685__sgt%2Fuq
   [junit4]   2> 422326 T1578 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52685__sgt%2Fuq
   [junit4]   2> 422329 T1607 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 422329 T1620 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 422329 T1600 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 422330 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 422337 T1621 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 422337 T1621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 422338 T1621 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 422603 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 422604 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:38359/_sgt/uq",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:38359__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 422608 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52685/_sgt/uq",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:52685__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 422608 T1601 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
   [junit4]   2> 422608 T1601 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 422612 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 422613 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 422612 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 423339 T1621 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 423339 T1621 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/collection1
   [junit4]   2> 423340 T1621 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 423341 T1621 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 423341 T1621 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 423342 T1621 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/collection1/'
   [junit4]   2> 423343 T1621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/collection1/lib/README' to classloader
   [junit4]   2> 423344 T1621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/collection1/lib/classes/' to classloader
   [junit4]   2> 423383 T1621 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 423433 T1621 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 423437 T1621 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 423448 T1621 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 423903 T1621 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 423904 T1621 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 423904 T1621 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 423911 T1621 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 423914 T1621 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 423926 T1621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 423931 T1621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 423935 T1621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 423936 T1621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 423937 T1621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 423937 T1621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 423938 T1621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 423939 T1621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 423939 T1621 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 423940 T1621 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1374158441458/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty1/
   [junit4]   2> 423940 T1621 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d90f5ce9
   [junit4]   2> 423941 T1621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty1
   [junit4]   2> 423941 T1621 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty1/index/
   [junit4]   2> 423942 T1621 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 423942 T1621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty1/index
   [junit4]   2> 423944 T1621 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6e1fcd3c lockFactory=org.apache.lucene.store.NativeFSLockFactory@dd4e647f)),segFN=segments_1,generation=1}
   [junit4]   2> 423944 T1621 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 423947 T1621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 423947 T1621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 423948 T1621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 423949 T1621 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 423949 T1621 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 423950 T1621 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 423950 T1621 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 423951 T1621 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 423951 T1621 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 423953 T1621 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 423955 T1621 oass.SolrIndexSearcher.<init> Opening Searcher@8cf092a5 main
   [junit4]   2> 423956 T1621 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 423956 T1621 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 423960 T1622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8cf092a5 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 423962 T1621 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 423962 T1621 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52685/_sgt/uq collection:collection1 shard:shard1
   [junit4]   2> 423962 T1621 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
   [junit4]   2> 423968 T1621 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 423973 T1621 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 423975 T1621 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 423976 T1621 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 423976 T1621 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52685/_sgt/uq/collection1/
   [junit4]   2> 423977 T1621 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 423977 T1621 oasc.SyncStrategy.syncToMe http://127.0.0.1:52685/_sgt/uq/collection1/ has no replicas
   [junit4]   2> 423977 T1621 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52685/_sgt/uq/collection1/ shard1
   [junit4]   2> 423978 T1621 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 424118 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 424125 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 424125 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 424125 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 424133 T1621 oasc.ZkController.register We are http://127.0.0.1:52685/_sgt/uq/collection1/ and leader is http://127.0.0.1:52685/_sgt/uq/collection1/
   [junit4]   2> 424134 T1621 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52685/_sgt/uq
   [junit4]   2> 424134 T1621 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 424135 T1621 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 424135 T1621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 424137 T1621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 424139 T1578 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 424140 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 424141 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 424259 T1578 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 424266 T1578 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43257
   [junit4]   2> 424267 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 424268 T1578 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 424269 T1578 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484
   [junit4]   2> 424270 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/'
   [junit4]   2> 424283 T1578 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/solr.xml
   [junit4]   2> 424337 T1578 oasc.CoreContainer.<init> New CoreContainer 1768612899
   [junit4]   2> 424338 T1578 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/]
   [junit4]   2> 424341 T1578 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 424342 T1578 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 424343 T1578 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 424344 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 424345 T1578 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 424346 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 424347 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 424348 T1578 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 424349 T1578 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 424350 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 424357 T1578 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 424358 T1578 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41008/solr
   [junit4]   2> 424359 T1578 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 424363 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 424364 T1634 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fa2fc6c name:ZooKeeperConnection Watcher:127.0.0.1:41008 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 424365 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 424366 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 424372 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 424374 T1636 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@beb0dba0 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 424374 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 424377 T1578 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 425380 T1578 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43257__sgt%2Fuq
   [junit4]   2> 425381 T1578 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43257__sgt%2Fuq
   [junit4]   2> 425383 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 425384 T1600 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 425384 T1607 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 425384 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 425385 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 425385 T1620 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 425384 T1636 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 425391 T1637 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 425392 T1637 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 425393 T1637 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 425630 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 425631 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52685/_sgt/uq",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:52685__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 425634 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:43257/_sgt/uq",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:43257__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 425635 T1601 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 425635 T1601 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 425639 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 425639 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 425639 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 425639 T1636 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 426393 T1637 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 426394 T1637 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/collection1
   [junit4]   2> 426394 T1637 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 426395 T1637 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 426395 T1637 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 426396 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/collection1/'
   [junit4]   2> 426397 T1637 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/collection1/lib/README' to classloader
   [junit4]   2> 426398 T1637 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/collection1/lib/classes/' to classloader
   [junit4]   2> 426432 T1637 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 426471 T1637 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 426472 T1637 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 426479 T1637 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 426869 T1637 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 426869 T1637 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 426870 T1637 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 426875 T1637 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 426878 T1637 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 426888 T1637 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 426891 T1637 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 426895 T1637 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 426896 T1637 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 426896 T1637 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 426897 T1637 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 426898 T1637 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 426898 T1637 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 426898 T1637 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 426899 T1637 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1374158444484/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty2/
   [junit4]   2> 426899 T1637 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d90f5ce9
   [junit4]   2> 426900 T1637 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty2
   [junit4]   2> 426900 T1637 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty2/index/
   [junit4]   2> 426900 T1637 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 426901 T1637 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty2/index
   [junit4]   2> 426902 T1637 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@97e7e55 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a35bab)),segFN=segments_1,generation=1}
   [junit4]   2> 426903 T1637 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 426905 T1637 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 426905 T1637 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 426906 T1637 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 426906 T1637 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 426907 T1637 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 426907 T1637 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 426908 T1637 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 426908 T1637 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 426908 T1637 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 426910 T1637 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 426912 T1637 oass.SolrIndexSearcher.<init> Opening Searcher@97079309 main
   [junit4]   2> 426912 T1637 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 426913 T1637 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 426916 T1638 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@97079309 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 426918 T1637 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 426918 T1637 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43257/_sgt/uq collection:collection1 shard:shard2
   [junit4]   2> 426918 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
   [junit4]   2> 426923 T1637 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 426927 T1637 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 426929 T1637 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 426929 T1637 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 426930 T1637 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43257/_sgt/uq/collection1/
   [junit4]   2> 426930 T1637 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 426930 T1637 oasc.SyncStrategy.syncToMe http://127.0.0.1:43257/_sgt/uq/collection1/ has no replicas
   [junit4]   2> 426931 T1637 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43257/_sgt/uq/collection1/ shard2
   [junit4]   2> 426931 T1637 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 427147 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 427153 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 427153 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 427154 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 427155 T1636 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 427188 T1637 oasc.ZkController.register We are http://127.0.0.1:43257/_sgt/uq/collection1/ and leader is http://127.0.0.1:43257/_sgt/uq/collection1/
   [junit4]   2> 427188 T1637 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43257/_sgt/uq
   [junit4]   2> 427189 T1637 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 427189 T1637 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 427189 T1637 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 427191 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 427193 T1578 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 427193 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 427195 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 427276 T1578 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 427281 T1578 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51362
   [junit4]   2> 427282 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 427283 T1578 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 427284 T1578 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539
   [junit4]   2> 427284 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/'
   [junit4]   2> 427293 T1578 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/solr.xml
   [junit4]   2> 427326 T1578 oasc.CoreContainer.<init> New CoreContainer -270370276
   [junit4]   2> 427327 T1578 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/]
   [junit4]   2> 427328 T1578 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 427329 T1578 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 427329 T1578 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 427330 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 427331 T1578 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 427331 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 427332 T1578 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 427333 T1578 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 427333 T1578 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 427334 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 427339 T1578 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 427340 T1578 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41008/solr
   [junit4]   2> 427340 T1578 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 427343 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 427344 T1650 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a155acba name:ZooKeeperConnection Watcher:127.0.0.1:41008 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 427345 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 427346 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 427353 T1578 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 427354 T1652 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69610b04 name:ZooKeeperConnection Watcher:127.0.0.1:41008/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 427355 T1578 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 427358 T1578 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 428362 T1578 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51362__sgt%2Fuq
   [junit4]   2> 428363 T1578 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51362__sgt%2Fuq
   [junit4]   2> 428366 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 428367 T1652 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 428367 T1600 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 428367 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428367 T1636 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 428368 T1607 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 428368 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428368 T1620 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 428370 T1636 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 428374 T1653 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 428374 T1653 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 428375 T1653 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 428658 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 428659 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:43257/_sgt/uq",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:43257__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 428663 T1601 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51362/_sgt/uq",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "node_name":"127.0.0.1:51362__sgt%2Fuq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 428664 T1601 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 428664 T1601 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 428667 T1620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428668 T1636 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428667 T1607 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428668 T1600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 428668 T1652 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 429376 T1653 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 429377 T1653 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/collection1
   [junit4]   2> 429377 T1653 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 429378 T1653 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 429378 T1653 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 429379 T1653 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/collection1/'
   [junit4]   2> 429380 T1653 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/collection1/lib/README' to classloader
   [junit4]   2> 429380 T1653 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/collection1/lib/classes/' to classloader
   [junit4]   2> 429406 T1653 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 429438 T1653 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 429439 T1653 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 429445 T1653 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 429770 T1653 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 429770 T1653 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 429771 T1653 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 429776 T1653 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 429778 T1653 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 429788 T1653 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 429792 T1653 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 429795 T1653 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 429797 T1653 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 429797 T1653 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 429797 T1653 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 429798 T1653 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 429799 T1653 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 429799 T1653 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 429800 T1653 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1374158447539/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty3/
   [junit4]   2> 429800 T1653 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d90f5ce9
   [junit4]   2> 429801 T1653 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty3
   [junit4]   2> 429801 T1653 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty3/index/
   [junit4]   2> 429801 T1653 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 429802 T1653 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1374158437984/jetty3/index
   [junit4]   2> 429803 T1653 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@303bf904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d8f5d7f)),segFN=segments_1,generation=1}
   [junit4]   2> 429804 T1653 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 429806 T1653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 429806 T1653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 429807 T1653 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 429807 T1653 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 429808 T1653 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 429808 T1653 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 429808 T1653 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 429809 T1653 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 429809 T1653 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 429811 T1653 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 429813 T1653 oass.SolrIndexSearcher.<init> Opening Searcher@c284477e main
   [junit4]   2> 429814 T1653 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 429814 T1653 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 429817 T1654 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c284477e main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 429819 T1653 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 429819 T1653 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51362/_sgt/uq collection:collection1 shard:shard1
   [junit4]   2> 429819 T1653 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
   [junit4]   2> 429826 T1653 oasc.ZkController.register We are http://127.0.0.1:51362/_sgt/uq/collection1/ and leader is http://127.0.0.1:52685/_sgt/uq/collection1/
   [junit4]   2> 429826 T1653 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51362/_sgt/uq
   [junit4]   2> 429826 T1653 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 429827 T1653 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> ASYNC  NEW_CORE C639 name=collection1 org.apache.solr.core.SolrCore@baa0de6d url=http://127.0.0.1:51362/_sgt/uq/collection1 node=127.0.0.1:51362__sgt%2Fuq C639_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:51362/_sgt/uq, state=down, node_name=127.0.0.1:51362__sgt%2Fuq, core=collection1}
   [junit4]   2> 429827 T1655 C639 P51362 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 429827 T1653 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 429828 T1655 C639 P51362 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 429828 T1655 C639 P51362 oasc.ZkController.publish publishing core=collection1 state=recovering
   [junit4]   2> 429829 T1655 C639 P51362 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 429829 T1578 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 429830 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 429829 T1655 C639 P51362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 429830 T1578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 429835 T1613 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
   [junit4]   2> 429899 T1578 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 429904 T1578 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51245
   [junit4]   2> 429904 T1578 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 429905 T1578 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 429906 T1578 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1374158450170
   [junit4]   2> 429906 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1374158450170/'
   [junit4]   2> 429918 T1578 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1374158450170/solr.xml
   [junit4]   2> 429962 T1578 oasc.CoreContainer.<init> New CoreContainer -2119900172
   [junit4]   2> 429963 T1578 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1374158450170/]
   [junit4]   2> 429964 T1578 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 429965 T1578 oashc.HttpShardHandlerFactory.getParameter Sett

[...truncated too long message...]

53298944)]} 0 1
   [junit4]   2> 1156094 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332155396096)]} 0 1
   [junit4]   2> 1156095 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332155396097)]} 0 1
   [junit4]   2> 1156095 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332156444672)]} 0 1
   [junit4]   2> 1156096 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332157493248)]} 0 1
   [junit4]   2> 1156097 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332158541824)]} 0 1
   [junit4]   2> 1156098 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332158541825)]} 0 1
   [junit4]   2> 1156098 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332159590400)]} 0 1
   [junit4]   2> 1156099 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332160638976)]} 0 1
   [junit4]   2> 1156114 T2981 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1440910332161687552)]} 0 1
   [junit4]   2> 1156116 T2981 C1391 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:500&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=1 
   [junit4]   2> 1156118 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332180561920)]} 0 1
   [junit4]   2> 1156119 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332181610496)]} 0 1
   [junit4]   2> 1156120 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332182659072)]} 0 1
   [junit4]   2> 1156121 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332183707648)]} 0 1
   [junit4]   2> 1156121 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332183707649)]} 0 1
   [junit4]   2> 1156122 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332184756224)]} 0 1
   [junit4]   2> 1156122 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332184756225)]} 0 1
   [junit4]   2> 1156123 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332185804800)]} 0 1
   [junit4]   2> 1156124 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332186853376)]} 0 1
   [junit4]   2> 1156124 T2981 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1440910332186853377)]} 0 1
   [junit4]   2> 1156125 T2981 C1391 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
   [junit4]   2> 1157118 T2990 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1157121 T2990 oass.SolrIndexSearcher.<init> Opening Searcher@4c449293 main
   [junit4]   2> 1157122 T2990 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1157123 T2989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c449293 main{StandardDirectoryReader(segments_3:23:nrt _6(4.5):c10/9)}
   [junit4]   2> 1157127 T2981 C1391 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:500&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
   [junit4]   2> 1157129 T2981 C1391 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
   [junit4]   2> 1157130 T2981 oas.SolrTestCaseJ4.tearDown ###Ending testCommitWithin
   [junit4]   2> 1157132 T2981 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 1157132 T2981 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1614986388
   [junit4]   2> 1157133 T2981 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@a321d06e
   [junit4]   2> 1157141 T2981 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=3,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=12,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=12,cumulative_deletesById=11,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 1157141 T2981 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1157142 T2981 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1157142 T2981 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1157146 T2981 C1391 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/index5083950844tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d7724ad)),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/index5083950844tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d7724ad)),segFN=segments_4,generation=4}
   [junit4]   2> 1157147 T2981 C1391 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> 1157148 T2981 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1157149 T2981 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1157149 T2981 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664;done=false>>]
   [junit4]   2> 1157149 T2981 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664
   [junit4]   2> 1157150 T2981 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664/index;done=false>>]
   [junit4]   2> 1157150 T2981 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1374159163664/index
   [junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_si=PostingsFormat(name=Memory doPackFST= false), range_facet_l=PostingsFormat(name=Memory doPackFST= false), intDefault=Pulsing41(freqCutoff=3 minBlockSize=20 maxBlockSize=42), range_facet_sl=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=Memory doPackFST= false), _version_=Pulsing41(freqCutoff=3 minBlockSize=20 maxBlockSize=42), multiDefault=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=MockRandom), subject=PostingsFormat(name=Memory doPackFST= false), text=PostingsFormat(name=Memory doPackFST= true), field_t=PostingsFormat(name=MockRandom)}, docValues:{}, sim=DefaultSimilarity, locale=hu_HU, timezone=Canada/Yukon
   [junit4]   2> NOTE: Linux 3.2.0-49-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=69539352,total=188022784
   [junit4]   2> NOTE: All tests run in this JVM: [TestClassNameShortening, TestLMDirichletSimilarityFactory, TestSurroundQueryParser, TestSolrQueryParser, HdfsCollectionsAPIDistributedZkTest, RecoveryZkTest, ShardSplitTest, ClusterStateUpdateTest, TestCoreContainer, TestFieldTypeCollectionResource, TestJoin, SoftAutoCommitTest, DirectUpdateHandlerOptimizeTest, OverseerCollectionProcessorTest, OverseerTest, TestSort, QueryParsingTest, AnalysisAfterCoreReloadTest, TestLFUCache, JsonLoaderTest, TestStressVersions, TestCollationKeyRangeQueries, TestGroupingSearch, BadCopyFieldTest, TestManagedSchema, FastVectorHighlighterTest, HdfsBasicDistributedZkTest, TestRandomDVFaceting, SearchHandlerTest, TestDynamicFieldResource, BasicDistributedZkTest, TestReloadAndDeleteDocs, TestSearchPerf, LukeRequestHandlerTest, ShardRoutingTest, StressHdfsTest, CoreAdminHandlerTest, AlternateDirectoryTest, TestStressRecovery, TestReplicationHandler, UpdateRequestProcessorFactoryTest, AutoCommitTest]
   [junit4] Completed on J0 in 13.44s, 3 tests, 1 error <<< FAILURES!

[...truncated 549 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:395: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:388: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1250: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:893: There were test failures: 319 suites, 1345 tests, 2 errors, 35 ignored (7 assumptions)

Total time: 46 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure