You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/26 05:37:39 UTC

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b94) - Build # 6313 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6313/
Java: 64bit/jdk1.8.0-ea-b94 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
shard1 is not consistent.  Got 87 from http://127.0.0.1:34231/collection1lastClient and got 75 from http://127.0.0.1:45893/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 87 from http://127.0.0.1:34231/collection1lastClient and got 75 from http://127.0.0.1:45893/collection1
	at __randomizedtesting.SeedInfo.seed([96044661F3C3E5FD:17E2C879849C85C1]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:491)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)


REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 is not consistent.  Got 305 from http://127.0.0.1:45955/collection1lastClient and got 251 from http://127.0.0.1:47597/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 305 from http://127.0.0.1:45955/collection1lastClient and got 251 from http://127.0.0.1:47597/collection1
	at __randomizedtesting.SeedInfo.seed([96044661F3C3E5FD:17E2C879849C85C1]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:491)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 9634 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4]   2> 303609 T1328 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 303613 T1328 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeySafeLeaderTest-1372217130517
[junit4:junit4]   2> 303613 T1328 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 303614 T1329 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 303714 T1328 oasc.ZkTestServer.run start zk server on port:51463
[junit4:junit4]   2> 303715 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 303718 T1335 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67041c71 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 303719 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 303719 T1328 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 303722 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 303723 T1337 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69c3eee1 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 303723 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 303724 T1328 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 303726 T1328 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 303728 T1328 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 303730 T1328 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 303732 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 303732 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 303736 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 303736 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 303738 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 303739 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 303741 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 303741 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 303743 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 303744 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 303746 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 303747 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 303749 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 303750 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 303752 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 303753 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 303755 T1328 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 303756 T1328 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 303830 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 303833 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52766
[junit4:junit4]   2> 303833 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 303834 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 303834 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663
[junit4:junit4]   2> 303834 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/solr.xml
[junit4:junit4]   2> 303834 T1328 oasc.CoreContainer.<init> New CoreContainer 1401282349
[junit4:junit4]   2> 303835 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/'
[junit4:junit4]   2> 303835 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/'
[junit4:junit4]   2> 303870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 303870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 303870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 303871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 303871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 303871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 303871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 303872 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 303872 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 303872 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 303875 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 303876 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 303876 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 303876 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 303878 T1348 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f8341de name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 303878 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 303880 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 303883 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 303885 T1350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2253bd46 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 303885 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 303886 T1328 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 303889 T1328 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 303891 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 303892 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52766_
[junit4:junit4]   2> 303893 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52766_
[junit4:junit4]   2> 303895 T1328 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 303900 T1328 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 303902 T1328 oasc.Overseer.start Overseer (id=89929621865693187-127.0.0.1:52766_-n_0000000000) starting
[junit4:junit4]   2> 303905 T1328 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 303908 T1352 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 303908 T1328 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 303911 T1328 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 303913 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 303915 T1351 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 303922 T1353 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 303922 T1353 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 305425 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 305426 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52766_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52766"}
[junit4:junit4]   2> 305426 T1351 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 305426 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 305431 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 305923 T1353 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/collection1
[junit4:junit4]   2> 305923 T1353 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 305924 T1353 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 305924 T1353 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 305926 T1353 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/collection1/'
[junit4:junit4]   2> 305926 T1353 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/collection1/lib/README' to classloader
[junit4:junit4]   2> 305927 T1353 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 305964 T1353 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 305999 T1353 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 306001 T1353 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 306004 T1353 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 306365 T1353 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 306370 T1353 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 306372 T1353 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 306384 T1353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 306386 T1353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 306388 T1353 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 306390 T1353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 306390 T1353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 306390 T1353 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 306391 T1353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 306391 T1353 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 306391 T1353 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 306397 T1353 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1372217130663/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data/
[junit4:junit4]   2> 306397 T1353 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 306397 T1353 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 306398 T1353 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data
[junit4:junit4]   2> 306398 T1353 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data/index/
[junit4:junit4]   2> 306398 T1353 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 306398 T1353 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data/index
[junit4:junit4]   2> 306401 T1353 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 306401 T1353 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 306403 T1353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 306403 T1353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 306403 T1353 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 306404 T1353 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 306404 T1353 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 306404 T1353 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 306404 T1353 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 306405 T1353 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 306405 T1353 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 306407 T1353 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 306411 T1353 oass.SolrIndexSearcher.<init> Opening Searcher@e84d741 main
[junit4:junit4]   2> 306411 T1353 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 306411 T1353 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 306414 T1354 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e84d741 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 306414 T1353 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 306414 T1353 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52766 collection:control_collection shard:shard1
[junit4:junit4]   2> 306415 T1353 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 306422 T1353 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 306424 T1353 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 306424 T1353 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 306424 T1353 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52766/collection1/
[junit4:junit4]   2> 306425 T1353 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 306425 T1353 oasc.SyncStrategy.syncToMe http://127.0.0.1:52766/collection1/ has no replicas
[junit4:junit4]   2> 306425 T1353 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52766/collection1/
[junit4:junit4]   2> 306425 T1353 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 306935 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 306944 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 306981 T1353 oasc.ZkController.register We are http://127.0.0.1:52766/collection1/ and leader is http://127.0.0.1:52766/collection1/
[junit4:junit4]   2> 306981 T1353 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52766
[junit4:junit4]   2> 306981 T1353 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 306981 T1353 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 306981 T1353 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 306983 T1353 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 306984 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 306984 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 306984 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 306987 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 306988 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 306990 T1357 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@319bd2e6 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 306990 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 306991 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 306993 T1328 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 307058 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 307060 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38575
[junit4:junit4]   2> 307061 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 307061 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 307061 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897
[junit4:junit4]   2> 307061 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/solr.xml
[junit4:junit4]   2> 307062 T1328 oasc.CoreContainer.<init> New CoreContainer 1430591718
[junit4:junit4]   2> 307062 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/'
[junit4:junit4]   2> 307062 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/'
[junit4:junit4]   2> 307098 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 307098 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 307098 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 307098 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 307099 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 307099 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 307099 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 307099 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 307099 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 307100 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 307103 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 307103 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 307103 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 307104 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 307105 T1368 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c4e7bb3 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 307106 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 307107 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 307110 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 307111 T1370 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37dc6819 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 307111 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 307115 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 308118 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38575_
[junit4:junit4]   2> 308120 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38575_
[junit4:junit4]   2> 308123 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 308123 T1357 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 308123 T1370 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 308124 T1350 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 308128 T1371 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 308128 T1371 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 308450 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 308451 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52766_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52766"}
[junit4:junit4]   2> 308455 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38575_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38575"}
[junit4:junit4]   2> 308455 T1351 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4]   2> 308455 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 308486 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 308486 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 308487 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 309129 T1371 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/collection1
[junit4:junit4]   2> 309129 T1371 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 309130 T1371 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 309130 T1371 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 309132 T1371 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/collection1/'
[junit4:junit4]   2> 309132 T1371 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/collection1/lib/README' to classloader
[junit4:junit4]   2> 309132 T1371 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 309155 T1371 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 309180 T1371 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 309182 T1371 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 309185 T1371 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 309470 T1371 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 309474 T1371 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 309476 T1371 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 309490 T1371 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 309492 T1371 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 309495 T1371 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 309496 T1371 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 309497 T1371 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 309497 T1371 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 309498 T1371 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 309498 T1371 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 309498 T1371 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 309498 T1371 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1372217133897/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/
[junit4:junit4]   2> 309498 T1371 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 309499 T1371 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 309499 T1371 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1
[junit4:junit4]   2> 309499 T1371 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index/
[junit4:junit4]   2> 309499 T1371 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 309500 T1371 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index
[junit4:junit4]   2> 309502 T1371 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 309503 T1371 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 309504 T1371 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 309504 T1371 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 309504 T1371 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 309505 T1371 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 309505 T1371 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 309505 T1371 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 309506 T1371 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 309506 T1371 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 309506 T1371 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 309508 T1371 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 309510 T1371 oass.SolrIndexSearcher.<init> Opening Searcher@12eb2495 main
[junit4:junit4]   2> 309511 T1371 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 309511 T1371 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 309513 T1372 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12eb2495 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 309514 T1371 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 309514 T1371 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38575 collection:collection1 shard:shard1
[junit4:junit4]   2> 309515 T1371 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 309521 T1371 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 309522 T1371 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 309523 T1371 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 309523 T1371 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38575/collection1/
[junit4:junit4]   2> 309523 T1371 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 309523 T1371 oasc.SyncStrategy.syncToMe http://127.0.0.1:38575/collection1/ has no replicas
[junit4:junit4]   2> 309523 T1371 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38575/collection1/
[junit4:junit4]   2> 309523 T1371 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 309995 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 310009 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 310009 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 310009 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 310031 T1371 oasc.ZkController.register We are http://127.0.0.1:38575/collection1/ and leader is http://127.0.0.1:38575/collection1/
[junit4:junit4]   2> 310031 T1371 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38575
[junit4:junit4]   2> 310031 T1371 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 310032 T1371 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 310032 T1371 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 310036 T1371 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 310037 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 310038 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 310038 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 310121 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 310123 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52428
[junit4:junit4]   2> 310123 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 310124 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 310124 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949
[junit4:junit4]   2> 310124 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/solr.xml
[junit4:junit4]   2> 310125 T1328 oasc.CoreContainer.<init> New CoreContainer 2058338414
[junit4:junit4]   2> 310125 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/'
[junit4:junit4]   2> 310126 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/'
[junit4:junit4]   2> 310164 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 310165 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 310165 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 310165 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 310165 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 310166 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 310166 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 310166 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 310166 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 310167 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 310169 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 310169 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 310170 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 310170 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 310172 T1384 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50c710ce name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 310172 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 310174 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 310178 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 310179 T1386 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@628d8cc8 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 310179 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 310184 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 311187 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52428_
[junit4:junit4]   2> 311189 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52428_
[junit4:junit4]   2> 311191 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 311191 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 311192 T1357 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 311192 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 311192 T1386 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 311193 T1370 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 311193 T1350 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 311198 T1387 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 311198 T1387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 311518 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 311519 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38575_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38575"}
[junit4:junit4]   2> 311525 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52428_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52428"}
[junit4:junit4]   2> 311525 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 311525 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 311530 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 311530 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 311530 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 311530 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 312199 T1387 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/collection1
[junit4:junit4]   2> 312199 T1387 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 312200 T1387 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 312200 T1387 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 312202 T1387 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/collection1/'
[junit4:junit4]   2> 312202 T1387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/collection1/lib/README' to classloader
[junit4:junit4]   2> 312202 T1387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 312224 T1387 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 312250 T1387 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 312251 T1387 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 312254 T1387 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 312530 T1387 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 312534 T1387 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 312536 T1387 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 312552 T1387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 312555 T1387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 312558 T1387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 312559 T1387 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 312560 T1387 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 312560 T1387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 312561 T1387 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 312561 T1387 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 312562 T1387 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 312562 T1387 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1372217136949/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/
[junit4:junit4]   2> 312562 T1387 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 312562 T1387 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 312562 T1387 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2
[junit4:junit4]   2> 312563 T1387 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index/
[junit4:junit4]   2> 312563 T1387 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 312563 T1387 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index
[junit4:junit4]   2> 312580 T1387 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 312580 T1387 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 312581 T1387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 312581 T1387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 312582 T1387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 312582 T1387 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 312582 T1387 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 312583 T1387 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 312583 T1387 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 312583 T1387 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 312583 T1387 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 312585 T1387 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 312587 T1387 oass.SolrIndexSearcher.<init> Opening Searcher@6b90bb82 main
[junit4:junit4]   2> 312588 T1387 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 312588 T1387 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 312591 T1388 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b90bb82 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 312592 T1387 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 312592 T1387 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52428 collection:collection1 shard:shard2
[junit4:junit4]   2> 312593 T1387 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 312600 T1387 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 312603 T1387 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 312603 T1387 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 312603 T1387 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52428/collection1/
[junit4:junit4]   2> 312603 T1387 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 312603 T1387 oasc.SyncStrategy.syncToMe http://127.0.0.1:52428/collection1/ has no replicas
[junit4:junit4]   2> 312603 T1387 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52428/collection1/
[junit4:junit4]   2> 312603 T1387 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 313037 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 313043 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 313043 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 313043 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 313044 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 313060 T1387 oasc.ZkController.register We are http://127.0.0.1:52428/collection1/ and leader is http://127.0.0.1:52428/collection1/
[junit4:junit4]   2> 313060 T1387 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52428
[junit4:junit4]   2> 313060 T1387 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 313060 T1387 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 313060 T1387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 313062 T1387 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 313063 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 313063 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 313063 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 313133 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 313136 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40019
[junit4:junit4]   2> 313136 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 313137 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 313137 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970
[junit4:junit4]   2> 313137 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/solr.xml
[junit4:junit4]   2> 313138 T1328 oasc.CoreContainer.<init> New CoreContainer 1837453187
[junit4:junit4]   2> 313138 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/'
[junit4:junit4]   2> 313139 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/'
[junit4:junit4]   2> 313204 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 313204 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 313205 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 313205 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 313205 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 313206 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 313206 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 313206 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 313207 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 313207 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 313211 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 313212 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 313212 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 313213 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 313215 T1400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33f6a5b1 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 313216 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 313217 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 313220 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 313221 T1402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78ed18c5 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 313221 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 313225 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 314228 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40019_
[junit4:junit4]   2> 314229 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40019_
[junit4:junit4]   2> 314232 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 314232 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 314232 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 314232 T1357 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 314233 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 314233 T1402 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 314234 T1350 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 314235 T1370 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 314235 T1386 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 314245 T1403 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 314245 T1403 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 314549 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 314550 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52428_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52428"}
[junit4:junit4]   2> 314555 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:40019_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40019"}
[junit4:junit4]   2> 314555 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 314555 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4]   2> 314559 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 314559 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 314559 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 314560 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 314560 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 315317 T1403 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/collection1
[junit4:junit4]   2> 315317 T1403 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 315318 T1403 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 315318 T1403 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 315319 T1403 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/collection1/'
[junit4:junit4]   2> 315320 T1403 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/collection1/lib/README' to classloader
[junit4:junit4]   2> 315320 T1403 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 315352 T1403 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 315392 T1403 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 315394 T1403 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 315398 T1403 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 315716 T1403 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 315722 T1403 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 315724 T1403 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 315736 T1403 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 315739 T1403 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 315741 T1403 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 315742 T1403 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 315743 T1403 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 315743 T1403 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 315744 T1403 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 315744 T1403 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 315744 T1403 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 315744 T1403 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1372217139970/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3/
[junit4:junit4]   2> 315744 T1403 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 315745 T1403 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 315745 T1403 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3
[junit4:junit4]   2> 315745 T1403 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3/index/
[junit4:junit4]   2> 315745 T1403 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 315746 T1403 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3/index
[junit4:junit4]   2> 315778 T1403 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 315778 T1403 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 315779 T1403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 315780 T1403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 315780 T1403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 315780 T1403 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 315781 T1403 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 315781 T1403 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 315781 T1403 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 315782 T1403 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 315782 T1403 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 315784 T1403 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 315788 T1403 oass.SolrIndexSearcher.<init> Opening Searcher@5c4f1361 main
[junit4:junit4]   2> 315789 T1403 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 315789 T1403 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 315792 T1404 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c4f1361 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 315793 T1403 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 315793 T1403 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40019 collection:collection1 shard:shard3
[junit4:junit4]   2> 315794 T1403 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4]   2> 315799 T1403 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 315805 T1403 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 315806 T1403 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 315806 T1403 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40019/collection1/
[junit4:junit4]   2> 315806 T1403 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 315806 T1403 oasc.SyncStrategy.syncToMe http://127.0.0.1:40019/collection1/ has no replicas
[junit4:junit4]   2> 315806 T1403 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40019/collection1/
[junit4:junit4]   2> 315807 T1403 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4]   2> 316066 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 316078 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 316078 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 316078 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 316078 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 316078 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 316115 T1403 oasc.ZkController.register We are http://127.0.0.1:40019/collection1/ and leader is http://127.0.0.1:40019/collection1/
[junit4:junit4]   2> 316115 T1403 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40019
[junit4:junit4]   2> 316115 T1403 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 316115 T1403 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 316115 T1403 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 316117 T1403 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 316118 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 316118 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 316119 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 316205 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 316215 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53163
[junit4:junit4]   2> 316216 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 316217 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 316217 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026
[junit4:junit4]   2> 316217 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/solr.xml
[junit4:junit4]   2> 316217 T1328 oasc.CoreContainer.<init> New CoreContainer 289448726
[junit4:junit4]   2> 316218 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/'
[junit4:junit4]   2> 316218 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/'
[junit4:junit4]   2> 316259 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 316260 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 316260 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 316260 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 316260 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 316261 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 316261 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 316261 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 316261 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 316262 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 316264 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 316265 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 316265 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 316266 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 316267 T1416 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a8aae73 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 316268 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 316269 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 316272 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 316273 T1418 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cbd8efe name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 316273 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 316276 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 317279 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53163_
[junit4:junit4]   2> 317281 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53163_
[junit4:junit4]   2> 317283 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 317283 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 317283 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 317284 T1402 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317284 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317284 T1357 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317285 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317286 T1418 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317286 T1350 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317286 T1386 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317287 T1370 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 317292 T1419 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 317293 T1419 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 317586 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 317587 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:40019_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40019"}
[junit4:junit4]   2> 317591 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53163_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53163"}
[junit4:junit4]   2> 317591 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 317591 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 317594 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317594 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317595 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317594 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317594 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 317595 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 318295 T1419 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/collection1
[junit4:junit4]   2> 318295 T1419 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 318296 T1419 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 318296 T1419 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 318297 T1419 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/collection1/'
[junit4:junit4]   2> 318298 T1419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/collection1/lib/README' to classloader
[junit4:junit4]   2> 318298 T1419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 318320 T1419 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 318353 T1419 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 318355 T1419 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 318359 T1419 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 318677 T1419 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 318682 T1419 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 318684 T1419 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 318697 T1419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 318701 T1419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 318704 T1419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 318706 T1419 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 318707 T1419 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 318707 T1419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 318710 T1419 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 318710 T1419 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 318710 T1419 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 318710 T1419 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1372217143026/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/
[junit4:junit4]   2> 318711 T1419 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 318711 T1419 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 318711 T1419 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4
[junit4:junit4]   2> 318712 T1419 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index/
[junit4:junit4]   2> 318712 T1419 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 318712 T1419 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index
[junit4:junit4]   2> 318716 T1419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 318716 T1419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 318718 T1419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 318718 T1419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 318718 T1419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 318719 T1419 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 318719 T1419 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 318719 T1419 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 318719 T1419 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 318720 T1419 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 318720 T1419 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 318721 T1419 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 318724 T1419 oass.SolrIndexSearcher.<init> Opening Searcher@27ac37d5 main
[junit4:junit4]   2> 318725 T1419 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 318725 T1419 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 318728 T1420 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27ac37d5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 318729 T1419 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 318729 T1419 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53163 collection:collection1 shard:shard1
[junit4:junit4]   2> 318732 T1419 oasc.ZkController.register We are http://127.0.0.1:53163/collection1/ and leader is http://127.0.0.1:38575/collection1/
[junit4:junit4]   2> 318732 T1419 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53163
[junit4:junit4]   2> 318732 T1419 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 318732 T1419 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C633 name=collection1 org.apache.solr.core.SolrCore@401eb8ea url=http://127.0.0.1:53163/collection1 node=127.0.0.1:53163_ C633_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:53163_, base_url=http://127.0.0.1:53163}
[junit4:junit4]   2> 318733 T1421 C633 P53163 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 318733 T1421 C633 P53163 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 318733 T1419 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 318733 T1421 C633 P53163 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 318733 T1421 C633 P53163 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 318734 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 318734 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 318734 T1421 C633 P53163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 318735 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 318738 T1366 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 318818 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 318821 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58905
[junit4:junit4]   2> 318822 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 318822 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 318822 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641
[junit4:junit4]   2> 318823 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/solr.xml
[junit4:junit4]   2> 318823 T1328 oasc.CoreContainer.<init> New CoreContainer 162768708
[junit4:junit4]   2> 318823 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/'
[junit4:junit4]   2> 318824 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/'
[junit4:junit4]   2> 318869 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 318869 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 318870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 318870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 318870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 318870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 318870 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 318871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 318871 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 318871 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 318874 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 318874 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 318875 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 318875 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 318877 T1433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bd834d1 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 318877 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 318878 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 318881 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 318883 T1435 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e26c672 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 318883 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 318888 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 319101 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 319102 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53163_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53163"}
[junit4:junit4]   2> 319106 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319106 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319106 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319106 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319106 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319106 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319107 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319739 T1366 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 319739 T1366 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:53163_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 319891 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58905_
[junit4:junit4]   2> 319893 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58905_
[junit4:junit4]   2> 319895 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319895 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319895 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319895 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 319895 T1357 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319896 T1435 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319896 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 319896 T1402 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319896 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 319896 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 319897 T1350 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319897 T1418 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319897 T1386 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319898 T1370 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 319901 T1436 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 319901 T1436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 320611 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 320612 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58905_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58905"}
[junit4:junit4]   2> 320612 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 320612 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 320616 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320616 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 320902 T1436 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/collection1
[junit4:junit4]   2> 320902 T1436 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 320903 T1436 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 320903 T1436 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 320905 T1436 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/collection1/'
[junit4:junit4]   2> 320905 T1436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/collection1/lib/README' to classloader
[junit4:junit4]   2> 320905 T1436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 320929 T1436 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 320956 T1436 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 320958 T1436 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 320961 T1436 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 321248 T1436 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 321253 T1436 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 321255 T1436 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 321267 T1436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 321269 T1436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 321273 T1436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 321274 T1436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 321275 T1436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 321275 T1436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 321276 T1436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 321276 T1436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 321276 T1436 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 321276 T1436 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1372217145641/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/
[junit4:junit4]   2> 321276 T1436 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 321277 T1436 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 321277 T1436 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5
[junit4:junit4]   2> 321277 T1436 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index/
[junit4:junit4]   2> 321277 T1436 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 321278 T1436 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index
[junit4:junit4]   2> 321280 T1436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 321281 T1436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 321282 T1436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 321282 T1436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 321283 T1436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 321283 T1436 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 321283 T1436 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 321284 T1436 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 321284 T1436 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 321284 T1436 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 321284 T1436 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 321286 T1436 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 321289 T1436 oass.SolrIndexSearcher.<init> Opening Searcher@7c113327 main
[junit4:junit4]   2> 321289 T1436 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 321289 T1436 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 321292 T1437 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7c113327 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 321294 T1436 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 321294 T1436 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58905 collection:collection1 shard:shard2
[junit4:junit4]   2> 321296 T1436 oasc.ZkController.register We are http://127.0.0.1:58905/collection1/ and leader is http://127.0.0.1:52428/collection1/
[junit4:junit4]   2> 321296 T1436 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58905
[junit4:junit4]   2> 321296 T1436 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 321296 T1436 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C634 name=collection1 org.apache.solr.core.SolrCore@69f91ff9 url=http://127.0.0.1:58905/collection1 node=127.0.0.1:58905_ C634_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:58905_, base_url=http://127.0.0.1:58905}
[junit4:junit4]   2> 321297 T1438 C634 P58905 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 321297 T1436 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 321297 T1438 C634 P58905 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 321298 T1438 C634 P58905 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 321298 T1438 C634 P58905 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 321298 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 321299 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 321299 T1438 C634 P58905 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 321299 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 321303 T1382 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 321370 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 321372 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36846
[junit4:junit4]   2> 321373 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 321373 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 321373 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205
[junit4:junit4]   2> 321374 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/solr.xml
[junit4:junit4]   2> 321374 T1328 oasc.CoreContainer.<init> New CoreContainer 1850708968
[junit4:junit4]   2> 321374 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/'
[junit4:junit4]   2> 321374 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/'
[junit4:junit4]   2> 321418 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 321419 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 321419 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 321419 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 321419 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 321419 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 321420 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 321420 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 321420 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 321420 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 321423 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 321424 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 321424 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 321425 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 321426 T1450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c7b7722 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 321427 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 321428 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 321431 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 321432 T1452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16ab4383 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 321432 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 321435 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C635 name=collection1 org.apache.solr.core.SolrCore@401eb8ea url=http://127.0.0.1:53163/collection1 node=127.0.0.1:53163_ C635_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:53163_, base_url=http://127.0.0.1:53163}
[junit4:junit4]   2> 321740 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:38575/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 321741 T1421 C635 P53163 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:53163 START replicas=[http://127.0.0.1:38575/collection1/] nUpdates=100
[junit4:junit4]   2> 321741 T1421 C635 P53163 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 321741 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 321742 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 321742 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 321742 T1421 C635 P53163 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 321742 T1421 C635 P53163 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:38575/collection1/. core=collection1
[junit4:junit4]   2> 321742 T1421 C635 P53163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C636 name=collection1 org.apache.solr.core.SolrCore@55437595 url=http://127.0.0.1:38575/collection1 node=127.0.0.1:38575_ C636_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38575_, base_url=http://127.0.0.1:38575, leader=true}
[junit4:junit4]   2> 321747 T1366 C636 P38575 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 321751 T1365 C636 P38575 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 321754 T1365 C636 P38575 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 321755 T1365 C636 P38575 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 321766 T1365 C636 P38575 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4]   2> 321767 T1365 C636 P38575 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 321767 T1365 C636 P38575 oass.SolrIndexSearcher.<init> Opening Searcher@37a6d5b6 realtime
[junit4:junit4]   2> 321767 T1365 C636 P38575 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 321768 T1365 C636 P38575 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4]   2> 321769 T1421 C635 P53163 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 321769 T1421 C635 P53163 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 321774 T1364 C636 P38575 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 321775 T1364 C636 P38575 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 321776 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 321776 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 321776 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 321778 T1366 C636 P38575 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 321778 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 321779 T1421 C635 P53163 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682
[junit4:junit4]   2> 321779 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6f72b084 fullCopy=false
[junit4:junit4]   2> 321782 T1365 C636 P38575 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 321784 T1421 C635 P53163 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 321798 T1421 C635 P53163 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 321798 T1421 C635 P53163 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 321800 T1421 C635 P53163 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4]   2> 321801 T1421 C635 P53163 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 321801 T1421 C635 P53163 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 321801 T1421 C635 P53163 oass.SolrIndexSearcher.<init> Opening Searcher@46db94a7 main
[junit4:junit4]   2> 321802 T1420 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46db94a7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 321802 T1421 C635 P53163 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682;done=true>>]
[junit4:junit4]   2> 321802 T1421 C635 P53163 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682
[junit4:junit4]   2> 321802 T1421 C635 P53163 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty4/index.20130626102548682
[junit4:junit4]   2> 321803 T1421 C635 P53163 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 321803 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 321803 T1421 C635 P53163 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 321803 T1421 C635 P53163 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 321805 T1421 C635 P53163 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 322121 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 322122 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58905_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58905"}
[junit4:junit4]   2> 322125 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53163_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53163"}
[junit4:junit4]   2> 322149 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322149 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322303 T1382 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 322303 T1382 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=6&state=recovering&nodeName=127.0.0.1:58905_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1000 
[junit4:junit4]   2> 322438 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36846_
[junit4:junit4]   2> 322440 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36846_
[junit4:junit4]   2> 322442 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322442 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322442 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322442 T1435 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322443 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 322442 T1452 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322442 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 322443 T1357 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322443 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 322443 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 322444 T1402 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322444 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 322445 T1350 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322445 T1370 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322446 T1386 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322446 T1418 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 322452 T1455 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 322452 T1455 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 323656 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 323657 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"7",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36846_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36846"}
[junit4:junit4]   2> 323657 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 323658 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4]   2> 323662 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323662 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 323663 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> ASYNC  NEW_CORE C637 name=collection1 org.apache.solr.core.SolrCore@69f91ff9 url=http://127.0.0.1:58905/collection1 node=127.0.0.1:58905_ C637_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:58905_, base_url=http://127.0.0.1:58905}
[junit4:junit4]   2> 324304 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52428/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 324304 T1438 C637 P58905 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58905 START replicas=[http://127.0.0.1:52428/collection1/] nUpdates=100
[junit4:junit4]   2> 324305 T1438 C637 P58905 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 324305 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 324306 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 324306 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 324306 T1438 C637 P58905 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 324306 T1438 C637 P58905 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52428/collection1/. core=collection1
[junit4:junit4]   2> 324306 T1438 C637 P58905 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C638 name=collection1 org.apache.solr.core.SolrCore@47fd0155 url=http://127.0.0.1:52428/collection1 node=127.0.0.1:52428_ C638_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:52428_, base_url=http://127.0.0.1:52428, leader=true}
[junit4:junit4]   2> 324315 T1381 C638 P52428 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 324316 T1380 C638 P52428 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5 
[junit4:junit4]   2> 324319 T1381 C638 P52428 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 324320 T1381 C638 P52428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 324326 T1381 C638 P52428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4]   2> 324327 T1381 C638 P52428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 324328 T1381 C638 P52428 oass.SolrIndexSearcher.<init> Opening Searcher@363bc23d realtime
[junit4:junit4]   2> 324329 T1381 C638 P52428 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 324329 T1381 C638 P52428 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 324330 T1438 C637 P58905 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 324330 T1438 C637 P58905 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 324333 T1382 C638 P52428 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 324333 T1382 C638 P52428 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 324334 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 324334 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 324334 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 324337 T1379 C638 P52428 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 324337 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 324338 T1438 C637 P58905 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242
[junit4:junit4]   2> 324339 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e0dfa4a fullCopy=false
[junit4:junit4]   2> 324342 T1380 C638 P52428 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 324344 T1438 C637 P58905 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 324357 T1438 C637 P58905 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 324358 T1438 C637 P58905 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 324360 T1438 C637 P58905 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index,segFN=segments_2,generation=2}
[junit4:junit4]   2> 324361 T1438 C637 P58905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 324361 T1438 C637 P58905 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 324362 T1438 C637 P58905 oass.SolrIndexSearcher.<init> Opening Searcher@7ddbd95e main
[junit4:junit4]   2> 324363 T1437 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ddbd95e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 324363 T1438 C637 P58905 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242;done=true>>]
[junit4:junit4]   2> 324363 T1438 C637 P58905 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242
[junit4:junit4]   2> 324363 T1438 C637 P58905 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty5/index.20130626102551242
[junit4:junit4]   2> 324364 T1438 C637 P58905 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 324364 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 324364 T1438 C637 P58905 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 324364 T1438 C637 P58905 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 324367 T1438 C637 P58905 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 324454 T1455 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/collection1
[junit4:junit4]   2> 324454 T1455 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 324455 T1455 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 324455 T1455 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 324456 T1455 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/collection1/'
[junit4:junit4]   2> 324456 T1455 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/collection1/lib/README' to classloader
[junit4:junit4]   2> 324457 T1455 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 324481 T1455 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 324508 T1455 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 324509 T1455 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 324516 T1455 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 324813 T1455 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 324819 T1455 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 324821 T1455 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 324835 T1455 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 324838 T1455 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 324840 T1455 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 324841 T1455 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 324841 T1455 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 324841 T1455 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 324842 T1455 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 324842 T1455 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 324842 T1455 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 324843 T1455 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1372217148205/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6/
[junit4:junit4]   2> 324843 T1455 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 324843 T1455 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 324843 T1455 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6
[junit4:junit4]   2> 324844 T1455 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6/index/
[junit4:junit4]   2> 324844 T1455 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 324844 T1455 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6/index
[junit4:junit4]   2> 324868 T1455 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty6/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 324868 T1455 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 324870 T1455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 324870 T1455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 324870 T1455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 324871 T1455 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 324871 T1455 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 324871 T1455 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 324873 T1455 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 324873 T1455 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 324873 T1455 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 324875 T1455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 324878 T1455 oass.SolrIndexSearcher.<init> Opening Searcher@7ad9f37a main
[junit4:junit4]   2> 324878 T1455 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 324878 T1455 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 324882 T1458 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ad9f37a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 324884 T1455 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 324884 T1455 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36846 collection:collection1 shard:shard3
[junit4:junit4]   2> 324887 T1455 oasc.ZkController.register We are http://127.0.0.1:36846/collection1/ and leader is http://127.0.0.1:40019/collection1/
[junit4:junit4]   2> 324887 T1455 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36846
[junit4:junit4]   2> 324888 T1455 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 324888 T1455 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C639 name=collection1 org.apache.solr.core.SolrCore@492fb3cf url=http://127.0.0.1:36846/collection1 node=127.0.0.1:36846_ C639_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:36846_, base_url=http://127.0.0.1:36846}
[junit4:junit4]   2> 324888 T1459 C639 P36846 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 324891 T1459 C639 P36846 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 324892 T1459 C639 P36846 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 324892 T1459 C639 P36846 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 324893 T1459 C639 P36846 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 324894 T1455 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 324895 T1328 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 324896 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 324896 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 324898 T1396 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 7, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 324968 T1328 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 324970 T1328 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34231
[junit4:junit4]   2> 324970 T1328 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 324971 T1328 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 324971 T1328 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802
[junit4:junit4]   2> 324971 T1328 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/solr.xml
[junit4:junit4]   2> 324971 T1328 oasc.CoreContainer.<init> New CoreContainer 1289269934
[junit4:junit4]   2> 324971 T1328 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/'
[junit4:junit4]   2> 324972 T1328 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/'
[junit4:junit4]   2> 325008 T1328 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 325009 T1328 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 325009 T1328 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 325009 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 325009 T1328 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 325010 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 325010 T1328 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 325011 T1328 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 325011 T1328 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 325011 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 325015 T1328 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 325015 T1328 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51463/solr
[junit4:junit4]   2> 325015 T1328 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 325016 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 325017 T1471 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5425b603 name:ZooKeeperConnection Watcher:127.0.0.1:51463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 325018 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 325019 T1328 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 325022 T1328 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 325023 T1473 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c9da2c1 name:ZooKeeperConnection Watcher:127.0.0.1:51463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 325023 T1328 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 325026 T1328 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 325168 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 325169 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58905_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58905"}
[junit4:junit4]   2> 325171 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"7",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36846_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36846"}
[junit4:junit4]   2> 325175 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325176 T1473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325176 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325175 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325175 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325175 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325176 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325176 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325176 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 325899 T1396 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 7, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 325899 T1396 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=7&state=recovering&nodeName=127.0.0.1:36846_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 326029 T1328 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34231_
[junit4:junit4]   2> 326030 T1328 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34231_
[junit4:junit4]   2> 326032 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 326032 T1473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 326032 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 326032 T1452 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326032 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326032 T1357 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326032 T1402 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326032 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 326033 T1435 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326032 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 326033 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326033 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326033 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326035 T1418 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326035 T1473 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326035 T1386 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326035 T1370 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326036 T1350 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 326040 T1474 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 326040 T1474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 326682 T1351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 326683 T1351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"8",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34231_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34231"}
[junit4:junit4]   2> 326683 T1351 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 326683 T1351 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 326688 T1357 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1386 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 326688 T1370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4]   2> 327042 T1474 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/collection1
[junit4:junit4]   2> 327042 T1474 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 327042 T1474 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 327043 T1474 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 327044 T1474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/collection1/'
[junit4:junit4]   2> 327044 T1474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/collection1/lib/README' to classloader
[junit4:junit4]   2> 327045 T1474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 327070 T1474 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 327115 T1474 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 327116 T1474 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 327121 T1474 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 327460 T1474 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 327465 T1474 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 327467 T1474 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 327479 T1474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 327484 T1474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 327489 T1474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 327491 T1474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 327491 T1474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 327491 T1474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 327492 T1474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 327493 T1474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 327493 T1474 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 327493 T1474 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1372217151802/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7/
[junit4:junit4]   2> 327493 T1474 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@48352a68
[junit4:junit4]   2> 327494 T1474 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 327494 T1474 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7
[junit4:junit4]   2> 327495 T1474 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7/index/
[junit4:junit4]   2> 327495 T1474 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 327495 T1474 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7/index
[junit4:junit4]   2> 327499 T1474 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1372217130517/jetty7/index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 327499 T1474 oasc.Sol

[...truncated too long message...]

           "leader":"true"}}}},
[junit4:junit4]   1>           "router":"compositeId"}}
[junit4:junit4]   1>   /solr/aliases.json (0)
[junit4:junit4]   1>   /solr/live_nodes (4)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:42490_ (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:45955_ (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:42917_ (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:47597_ (0)
[junit4:junit4]   1>   /solr/overseer (3)
[junit4:junit4]   1>   DATA:
[junit4:junit4]   1>       
[junit4:junit4]   1>    /solr/overseer/queue (0)
[junit4:junit4]   1>    /solr/overseer/queue-work (0)
[junit4:junit4]   1>    /solr/overseer/collection-queue-work (0)
[junit4:junit4]   1>   /solr/collections (2)
[junit4:junit4]   1>    /solr/collections/collection1 (3)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"configName":"conf1"}
[junit4:junit4]   1>     /solr/collections/collection1/shards (0)
[junit4:junit4]   1>     /solr/collections/collection1/leader_elect (1)
[junit4:junit4]   1>      /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4]   1>       /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/89929625827278858-4-n_0000000002 (0)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/89929625827278860-5-n_0000000003 (0)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/89929625827278862-2-n_0000000004 (0)
[junit4:junit4]   1>     /solr/collections/collection1/leaders (1)
[junit4:junit4]   1>      /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"collection1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:47597_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:47597"}
[junit4:junit4]   1>    /solr/collections/control_collection (3)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"configName":"conf1"}
[junit4:junit4]   1>     /solr/collections/control_collection/shards (0)
[junit4:junit4]   1>     /solr/collections/control_collection/leader_elect (1)
[junit4:junit4]   1>      /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4]   1>       /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4]   1>        /solr/collections/control_collection/leader_elect/shard1/election/89929625827278851-1-n_0000000000 (0)
[junit4:junit4]   1>     /solr/collections/control_collection/leaders (1)
[junit4:junit4]   1>      /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"collection1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:42917_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:42917"}
[junit4:junit4]   1>   /solr/overseer_elect (2)
[junit4:junit4]   1>    /solr/overseer_elect/election (4)
[junit4:junit4]   1>     /solr/overseer_elect/election/89929625827278860-127.0.0.1:42490_-n_0000000004 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89929625827278862-127.0.0.1:45955_-n_0000000005 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89929625827278858-127.0.0.1:47597_-n_0000000003 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89929625827278851-127.0.0.1:42917_-n_0000000000 (0)
[junit4:junit4]   1>    /solr/overseer_elect/leader (0)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"id":"89929625827278851-127.0.0.1:42917_-n_0000000000"}
[junit4:junit4]   1>  /zookeeper (1)
[junit4:junit4]   1>  DATA:
[junit4:junit4]   1>      
[junit4:junit4]   1> 
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=96044661F3C3E5FD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_TN -Dtests.timezone=America/Indiana/Vincennes -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE  101s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: shard1 is not consistent.  Got 305 from http://127.0.0.1:45955/collection1lastClient and got 251 from http://127.0.0.1:47597/collection1
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([96044661F3C3E5FD:17E2C879849C85C1]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   2> 464930 T1524 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 100840 T1523 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4]   2> 465047 T1599 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 465047 T1599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 465047 T1599 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 466131 T1616 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 466131 T1616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 466131 T1616 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Memory doPackFST= false), text=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=Memory doPackFST= false), rnd_b=Pulsing41(freqCutoff=10 minBlockSize=36 maxBlockSize=118), intDefault=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=Memory doPackFST= false), id=Pulsing41(freqCutoff=10 minBlockSize=36 maxBlockSize=118), a_t=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=NestedPulsing), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=NestedPulsing), a_si=PostingsFormat(name=NestedPulsing)}, docValues:{timestamp=DocValuesFormat(name=SimpleText)}, sim=DefaultSimilarity, locale=ar_TN, timezone=America/Indiana/Vincennes
[junit4:junit4]   2> NOTE: Linux 3.2.0-48-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=1,free=152233376,total=354418688
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrCoreCheckLockOnStartupTest, DOMUtilTest, DefaultValueUpdateProcessorTest, OpenExchangeRatesOrgProviderTest, SolrIndexConfigTest, CopyFieldTest, SuggesterWFSTTest, TestReload, TestQueryTypes, TestFoldingMultitermQuery, TestDistributedSearch, FieldAnalysisRequestHandlerTest, TestBinaryResponseWriter, TestSchemaNameResource, CollectionsAPIDistributedZkTest, DirectSolrSpellCheckerTest, CoreContainerCoreInitFailuresTest, BadIndexSchemaTest, StatsComponentTest, TestCSVLoader, TestHashPartitioner, TestSolrXml, TestMaxScoreQueryParser, TestPropInject, TestAnalyzedSuggestions, TestBM25SimilarityFactory, SuggesterTSTTest, TermVectorComponentDistributedTest, TestFieldTypeResource, SolrCmdDistributorTest, TestDynamicFieldCollectionResource, SpellPossibilityIteratorTest, TestSerializedLuceneMatchVersion, SignatureUpdateProcessorFactoryTest, ConvertedLegacyTest, BasicDistributedZk2Test, TestDynamicFieldResource, NumericFieldsTest, MultiTermTest, TestSchemaVersionResource, HighlighterTest, TestJmxMonitoredMap, TestBinaryField, ZkSolrClientTest, HardAutoCommitTest, PingRequestHandlerTest, IndexReaderFactoryTest, TestStressReorder, XmlUpdateRequestHandlerTest, TestWordDelimiterFilterFactory, SolrTestCaseJ4Test, DateMathParserTest, TestNumberUtils, DirectUpdateHandlerTest, TimeZoneUtilsTest, XsltUpdateRequestHandlerTest, OpenCloseCoreStressTest, TestShardHandlerFactory, SolrPluginUtilsTest, HighlighterConfigTest, TestMergePolicyConfig, CurrencyFieldXmlFileTest, FileUtilsTest, TestSolrJ, DistanceFunctionTest, ZkNodePropsTest, TestZkChroot, TestUpdate, TestRecovery, BasicDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 102.12s, 1 test, 1 failure <<< FAILURES!

[...truncated 451 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:366: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1243: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 2 failures, 16 ignored (7 assumptions)

Total time: 34 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b94 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build # 6314 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6314/
Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

1 tests failed.
REGRESSION:  org.apache.solr.update.AutoCommitTest.testMaxTime

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C8BBB816E66B29B6:524FC5F478F1B58A]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:530)
	at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:227)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1265</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1438878910409342976</long><int name="intDefault">42</int><date name="timestamp">2013-06-26T04:44:21.276Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
</response>

	request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:523)
	... 41 more




Build Log:
[...truncated 9329 lines...]
[junit4:junit4] Suite: org.apache.solr.update.AutoCommitTest
[junit4:junit4]   2> 109553 T306 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568
[junit4:junit4]   2> 109554 T306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 109555 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 109555 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 109590 T306 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 109640 T306 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 109640 T306 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 109647 T306 oass.IndexSchema.readSchema [null] Schema name=test
[junit4:junit4]   2> 109996 T306 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 110002 T306 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 110005 T306 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 110011 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 110015 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 110019 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 110019 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 110020 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 110020 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 110020 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 110021 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 110021 T306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 110022 T306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 110022 T306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 110029 T306 oasc.CoreContainer.<init> New CoreContainer -758671123
[junit4:junit4]   2> 110061 T306 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 110062 T306 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/
[junit4:junit4]   2> 110062 T306 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4:junit4]   2> 110063 T306 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 110063 T306 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568
[junit4:junit4]   2> 110064 T306 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index/
[junit4:junit4]   2> 110064 T306 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 110065 T306 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index
[junit4:junit4]   2> 110066 T306 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_1,generation=1}
[junit4:junit4]   2> 110067 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 110067 T306 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 110068 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 110069 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 110069 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 110070 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 110070 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 110071 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 110071 T306 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 110071 T306 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 110072 T306 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 110072 T306 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 110073 T306 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 110073 T306 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 110073 T306 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 110074 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 110074 T306 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 110075 T306 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 110075 T306 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110075 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110076 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110076 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110077 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110077 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110077 T306 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110078 T306 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110078 T306 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 110079 T306 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 110080 T306 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 110082 T306 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 110083 T306 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [SOLRCONFIG-MANAGED-SCHEMA.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SOLRCONFIG-MANAGED-SCHEMA-NAMED-SCHEMA.XML.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SCHEMA-BEHAVIOR.XML, SOLRCONFIG-MERGEPOLICY-DEFAULTS.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, STOP-2.TXT, SCHEMA11.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, SCHEMA15.XML, BAD-SOLRCONFIG-UNEXPECTED-SCHEMA-ATTRIBUTE.XML, SOLRCONFIG-NATIVELOCK.XML, SCHEMA-SIM.XML, CONDITIONAL.UPDATEPROCESSOR.JS, SCHEMA-DOCVALUES.XML, SCHEMA-ID-AND-VERSION-FIELDS-ONLY.XML, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-DOCVALUES-NOT-REQUIRED-NO-DEFAULT.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, SCHEMA-EFF.XML, SOLRCONFIG_CODEC.XML, SOLRCONFIG-MUTABLE-MANAGED-SCHEMA.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML, HYPHENATION.DTD, TRIVIAL.UPDATEPROCESSOR1.JS, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-PHRASESUGGEST.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-BASIC.XML, SCHEMA-LMJELINEKMERCER.XML, BAD-MP-SOLRCONFIG.XML, JASUGGEST.TXT, BAD-SCHEMA-DUP-FIELD.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SOLRCONFIG-MASTER.XML, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-TFIDF.XML, SCHEMA-REPLICATION1.XML, STOPWITHBOM.TXT, SOLRCONFIG-ELEVATE.XML, SYNONYMS.TXT, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-COMPONENTS-NAME.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, WDFTYPES.TXT, BAD-SCHEMA-CURRENCY-FT-OER-NORATES.XML, SOLRCONFIG-DELPOLICY1.XML, BAD_SOLRCONFIG.XML, BAD-SCHEMA-UNSUPPORTED-DOCVALUES.XML, HUNSPELL-TEST.DIC, SCHEMA-NUMERIC.XML, SCHEMA-LUCENEMATCHVERSION.XML, KEEP-2.TXT, SCHEMA-STOP-KEEP.XML, SCHEMA-IB.XML, STOP-SNOWBALL.TXT, SCHEMA-ONE-FIELD-NO-DYNAMIC-FIELD.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-DOCVALUESMULTI.XML, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG-SLAVE.XML, SCHEMA-CLASS-NAME-SHORTENING-ON-SERIALIZATION.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, STOPWORDSWRONGENCODING.TXT, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, STOPTYPES-1.TXT, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SOLRCONFIG.SNIPPET.RANDOMINDEXCONFIG.XML, REGEX-BOOST-PROCESSOR-TEST.TXT, SOLRCONFIG-MINIMAL.XML, SCHEMA-DOCVALUESFACETING.XML, SOLRCONFIG-TLOG.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-POSTINGSHIGHLIGHT.XML, DA_COMPOUNDDICTIONARY.TXT, SOLRCONFIG-TERMINDEX.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, BAD-SCHEMA-NON-GLOB-COPYFIELD-SOURCE-MATCHING-NOTHING-SHOULD-FAIL-TEST.XML, SCHEMASURROUND.XML, OPEN-EXCHANGE-RATES.JSON, BAD-SCHEMA-DUP-FIELDTYPE.XML, PHRASESUGGEST.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-MASTER2.XML, OLD_SYNONYMS.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SCHEMA-LMDIRICHLET.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-FOLDING.XML, SOLRCONFIG-CACHING.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD-CURRENCY.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SCHEMA-BM25.XML, SOLRCONFIG-ENABLEPLUGIN.XML, STOP-1.TXT, BAD-SCHEMA-SWEETSPOT-BOTH-TF.XML, SCHEMA-REST.XML, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SCHEMA12.XML, SOLRCONFIG-TRANSFORMERS.XML, SCHEMA-COPYFIELD-TEST.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-BASELINE.XML, CURRENCY.XML, DA_UTF8.XML, FUZZYSUGGEST.TXT, SCHEMA-POSTINGSHIGHLIGHT.XML, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SOLRCONFIG-SOLR-749.XML, STOPWORDS.TXT, BAD-SCHEMA-BOGUS-ANALYSIS-PARAMETERS.XML, HUNSPELL-TEST.AFF, SOLRCONFIG-TLOG-MANAGED-SCHEMA.XML, SOLRCONFIG_PERF.XML, SCHEMA-SWEETSPOT.XML, SOLRCONFIG-SIMPLELOCK.XML, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-TINY.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-NORMS.XML, FRENCHARTICLES.TXT, SOLRCONFIG-LAZYWRITER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SCHEMA-REPLICATION2.XML, SCHEMA-SNIPPET-TYPE.XML, SCHEMA-SPATIAL.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-HYPERBOLIC.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-XINCLUDE.XML, XSLT, SOLRCONFIG-WARMER.XML, SOLRCONFIG-DELPOLICY2.XML, KEEP-1.TXT, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA-TRIE.XML, SOLRCONFIG-TLOG-MUTABLE-MANAGED-SCHEMA.XML, SCHEMA-BINARYFIELD.XML, SCHEMA-REST-LUCENE-MATCH-VERSION.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-CODE-IN-XML.XML, SCHEMA-MINIMAL.XML, STOPTYPES-2.TXT, SCHEMA_CODEC.XML, STEMDICT.TXT, SCHEMA-SYNONYM-TOKENIZER.XML, ELEVATE.XML, SCHEMA-COLLATE.XML, PROTWORDS.TXT, SOLRCONFIG-DEFAULTS.XML, SCHEMA-XINCLUDE.XML, SCHEMA-SNIPPET-FIELD.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMA-PHRASESUGGEST.XML, SCHEMA-DFR.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SOLRCONFIG-MASTER1.XML, SOLRCONFIG-FUNCTIONQUERY.XML, BAD-SOLRCONFIG-SCHEMA-MUTABLE-BUT-NOT-MANAGED.XML, SOLRCONFIG-MASTER3.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, SOLRCONFIG-ALTDIRECTORY.XML, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-DEFAULT-CODE.XML, SOLRCONFIG-MERGEPOLICY-LEGACY.XML, BAD-SCHEMA-BOGUS-FIELD-PARAMETERS.XML]
[junit4:junit4]   2> 110085 T306 oass.SolrIndexSearcher.<init> Opening Searcher@aa0f811b main
[junit4:junit4]   2> 110085 T306 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/tlog
[junit4:junit4]   2> 110086 T306 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 110087 T306 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 110087 T306 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 110158 T306 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 110178 T307 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@aa0f811b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 110178 T306 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 110180 T306 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 110185 T306 oas.SolrTestCaseJ4.setUp ###Starting testMaxTime
[junit4:junit4]   2> ASYNC  NEW_CORE C413 name=collection1 org.apache.solr.core.SolrCore@2ea158f0
[junit4:junit4]   2> 110194 T306 C413 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_1,generation=1}
[junit4:junit4]   2> 110195 T306 C413 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 110196 T306 C413 oass.SolrIndexSearcher.<init> Opening Searcher@4828723b realtime
[junit4:junit4]   2> 110197 T306 C413 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1438878908239839232)} 0 5
[junit4:junit4]   2> 110202 T306 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 110203 T306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 110203 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 110204 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 110229 T306 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 110266 T306 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 110267 T306 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 110270 T306 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 110651 T306 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 110657 T306 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 110660 T306 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 110667 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 110670 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 110673 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 110674 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 110674 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 110675 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 110675 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 110676 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 110676 T306 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/
[junit4:junit4]   2> 110676 T306 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4:junit4]   2> 110677 T306 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index/
[junit4:junit4]   2> 110677 T306 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 110678 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 110679 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 110679 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 110680 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 110680 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 110681 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 110681 T306 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 110682 T306 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 110682 T306 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 110682 T306 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 110683 T306 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 110683 T306 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 110684 T306 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 110684 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 110684 T306 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 110685 T306 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 110685 T306 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110685 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110686 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110686 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110687 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110687 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110687 T306 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110688 T306 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 110688 T306 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 110689 T306 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 110690 T306 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 110691 T306 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 110693 T306 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [SOLRCONFIG-MANAGED-SCHEMA.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SOLRCONFIG-MANAGED-SCHEMA-NAMED-SCHEMA.XML.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SCHEMA-BEHAVIOR.XML, SOLRCONFIG-MERGEPOLICY-DEFAULTS.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, STOP-2.TXT, SCHEMA11.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, SCHEMA15.XML, BAD-SOLRCONFIG-UNEXPECTED-SCHEMA-ATTRIBUTE.XML, SOLRCONFIG-NATIVELOCK.XML, SCHEMA-SIM.XML, CONDITIONAL.UPDATEPROCESSOR.JS, SCHEMA-DOCVALUES.XML, SCHEMA-ID-AND-VERSION-FIELDS-ONLY.XML, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-DOCVALUES-NOT-REQUIRED-NO-DEFAULT.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, SCHEMA-EFF.XML, SOLRCONFIG_CODEC.XML, SOLRCONFIG-MUTABLE-MANAGED-SCHEMA.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML, HYPHENATION.DTD, TRIVIAL.UPDATEPROCESSOR1.JS, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-PHRASESUGGEST.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-BASIC.XML, SCHEMA-LMJELINEKMERCER.XML, BAD-MP-SOLRCONFIG.XML, JASUGGEST.TXT, BAD-SCHEMA-DUP-FIELD.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SOLRCONFIG-MASTER.XML, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-TFIDF.XML, SCHEMA-REPLICATION1.XML, STOPWITHBOM.TXT, SOLRCONFIG-ELEVATE.XML, SYNONYMS.TXT, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-COMPONENTS-NAME.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, WDFTYPES.TXT, BAD-SCHEMA-CURRENCY-FT-OER-NORATES.XML, SOLRCONFIG-DELPOLICY1.XML, BAD_SOLRCONFIG.XML, BAD-SCHEMA-UNSUPPORTED-DOCVALUES.XML, HUNSPELL-TEST.DIC, SCHEMA-NUMERIC.XML, SCHEMA-LUCENEMATCHVERSION.XML, KEEP-2.TXT, SCHEMA-STOP-KEEP.XML, SCHEMA-IB.XML, STOP-SNOWBALL.TXT, SCHEMA-ONE-FIELD-NO-DYNAMIC-FIELD.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-DOCVALUESMULTI.XML, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG-SLAVE.XML, SCHEMA-CLASS-NAME-SHORTENING-ON-SERIALIZATION.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, STOPWORDSWRONGENCODING.TXT, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, STOPTYPES-1.TXT, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SOLRCONFIG.SNIPPET.RANDOMINDEXCONFIG.XML, REGEX-BOOST-PROCESSOR-TEST.TXT, SOLRCONFIG-MINIMAL.XML, SCHEMA-DOCVALUESFACETING.XML, SOLRCONFIG-TLOG.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-POSTINGSHIGHLIGHT.XML, DA_COMPOUNDDICTIONARY.TXT, SOLRCONFIG-TERMINDEX.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, BAD-SCHEMA-NON-GLOB-COPYFIELD-SOURCE-MATCHING-NOTHING-SHOULD-FAIL-TEST.XML, SCHEMASURROUND.XML, OPEN-EXCHANGE-RATES.JSON, BAD-SCHEMA-DUP-FIELDTYPE.XML, PHRASESUGGEST.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-MASTER2.XML, OLD_SYNONYMS.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SCHEMA-LMDIRICHLET.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-FOLDING.XML, SOLRCONFIG-CACHING.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD-CURRENCY.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SCHEMA-BM25.XML, SOLRCONFIG-ENABLEPLUGIN.XML, STOP-1.TXT, BAD-SCHEMA-SWEETSPOT-BOTH-TF.XML, SCHEMA-REST.XML, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SCHEMA12.XML, SOLRCONFIG-TRANSFORMERS.XML, SCHEMA-COPYFIELD-TEST.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-BASELINE.XML, CURRENCY.XML, DA_UTF8.XML, FUZZYSUGGEST.TXT, SCHEMA-POSTINGSHIGHLIGHT.XML, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SOLRCONFIG-SOLR-749.XML, STOPWORDS.TXT, BAD-SCHEMA-BOGUS-ANALYSIS-PARAMETERS.XML, HUNSPELL-TEST.AFF, SOLRCONFIG-TLOG-MANAGED-SCHEMA.XML, SOLRCONFIG_PERF.XML, SCHEMA-SWEETSPOT.XML, SOLRCONFIG-SIMPLELOCK.XML, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-TINY.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-NORMS.XML, FRENCHARTICLES.TXT, SOLRCONFIG-LAZYWRITER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SCHEMA-REPLICATION2.XML, SCHEMA-SNIPPET-TYPE.XML, SCHEMA-SPATIAL.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-HYPERBOLIC.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-XINCLUDE.XML, XSLT, SOLRCONFIG-WARMER.XML, SOLRCONFIG-DELPOLICY2.XML, KEEP-1.TXT, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA-TRIE.XML, SOLRCONFIG-TLOG-MUTABLE-MANAGED-SCHEMA.XML, SCHEMA-BINARYFIELD.XML, SCHEMA-REST-LUCENE-MATCH-VERSION.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-CODE-IN-XML.XML, SCHEMA-MINIMAL.XML, STOPTYPES-2.TXT, SCHEMA_CODEC.XML, STEMDICT.TXT, SCHEMA-SYNONYM-TOKENIZER.XML, ELEVATE.XML, SCHEMA-COLLATE.XML, PROTWORDS.TXT, SOLRCONFIG-DEFAULTS.XML, SCHEMA-XINCLUDE.XML, SCHEMA-SNIPPET-FIELD.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMA-PHRASESUGGEST.XML, SCHEMA-DFR.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SOLRCONFIG-MASTER1.XML, SOLRCONFIG-FUNCTIONQUERY.XML, BAD-SOLRCONFIG-SCHEMA-MUTABLE-BUT-NOT-MANAGED.XML, SOLRCONFIG-MASTER3.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, SOLRCONFIG-ALTDIRECTORY.XML, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-DEFAULT-CODE.XML, SOLRCONFIG-MERGEPOLICY-LEGACY.XML, BAD-SCHEMA-BOGUS-FIELD-PARAMETERS.XML]
[junit4:junit4]   2> 110694 T306 oass.SolrIndexSearcher.<init> Opening Searcher@8d67b6f main
[junit4:junit4]   2> 110695 T306 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 110695 T306 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 110696 T306 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 110697 T306 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 110703 T308 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8d67b6f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 110703 T306 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 110704 T306 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 110705 T306 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4]   2> 110707 T306 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_1,generation=1}
[junit4:junit4]   2> 110707 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 110708 T306 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 110708 T306 oass.SolrIndexSearcher.<init> Opening Searcher@10def6eb main
[junit4:junit4]   2> 110709 T306 oasc.CoreContainer.registerCore replacing core: collection1
[junit4:junit4]   2> 110710 T306 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2ea158f0
[junit4:junit4]   2> 110709 T308 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10def6eb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 110710 T306 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 110711 T306 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 112262 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[529 (1438878910409342976)]} 0 1548
[junit4:junit4]   2> 113262 T309 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4]   2> 113266 T309 oass.SolrIndexSearcher.<init> Opening Searcher@b9e805b5 main
[junit4:junit4]   2> 113267 T309 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 113267 T308 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b9e805b5 main{StandardDirectoryReader(segments_1:3:nrt _0(5.0):C1)}
[junit4:junit4]   2> ASYNC  NEW_CORE C414 name=collection1 org.apache.solr.core.SolrCore@4cf9e62
[junit4:junit4]   2> 113528 T306 C414 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:529&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=1265 
[junit4:junit4]   2> 113693 T306 oas.SolrTestCaseJ4.assertQ ERROR REQUEST FAILED: xpath=//result[@numFound=0]
[junit4:junit4]   2> 		xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4:junit4]   2> 	<response>
[junit4:junit4]   2> 	<lst name="responseHeader"><int name="status">0</int><int name="QTime">1265</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1438878910409342976</long><int name="intDefault">42</int><date name="timestamp">2013-06-26T04:44:21.276Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
[junit4:junit4]   2> 	</response>
[junit4:junit4]   2> 	
[junit4:junit4]   2> 		request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
[junit4:junit4]   2> 113694 T306 oasc.SolrException.log ERROR REQUEST FAILED: q=id:529&start=0&qt=standard&version=2.2&rows=20:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
[junit4:junit4]   2> 		xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4:junit4]   2> 	<response>
[junit4:junit4]   2> 	<lst name="responseHeader"><int name="status">0</int><int name="QTime">1265</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1438878910409342976</long><int name="intDefault">42</int><date name="timestamp">2013-06-26T04:44:21.276Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
[junit4:junit4]   2> 	</response>
[junit4:junit4]   2> 	
[junit4:junit4]   2> 		request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
[junit4:junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:523)
[junit4:junit4]   2> 		at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:227)
[junit4:junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
[junit4:junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[junit4:junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:613)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:780)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 113695 T306 oas.SolrTestCaseJ4.tearDown ###Ending testMaxTime
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AutoCommitTest -Dtests.method=testMaxTime -Dtests.seed=C8BBB816E66B29B6 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv_LV_PREEURO -Dtests.timezone=Atlantic/Jan_Mayen -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   3.52s J0 | AutoCommitTest.testMaxTime <<<
[junit4:junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C8BBB816E66B29B6:524FC5F478F1B58A]:0)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:530)
[junit4:junit4]    > 	at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:227)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:780)
[junit4:junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
[junit4:junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4:junit4]    > <response>
[junit4:junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">1265</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1438878910409342976</long><int name="intDefault">42</int><date name="timestamp">2013-06-26T04:44:21.276Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
[junit4:junit4]    > </response>
[junit4:junit4]    > 	request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:523)
[junit4:junit4]    > 	... 41 more
[junit4:junit4]   2> 113705 T306 oas.SolrTestCaseJ4.setUp ###Starting testMaxDocs
[junit4:junit4]   2> 113711 T306 C414 oass.SolrIndexSearcher.<init> Opening Searcher@8c0f6595 realtime
[junit4:junit4]   2> 113711 T306 C414 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1438878911928729600)} 0 1
[junit4:junit4]   2> 113713 T306 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 113713 T306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 113714 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 113714 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 113741 T306 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 113778 T306 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 113778 T306 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 113783 T306 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 114213 T306 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 114220 T306 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 114223 T306 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 114229 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 114232 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 114236 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 114237 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 114237 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 114238 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 114238 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 114238 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 114239 T306 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/
[junit4:junit4]   2> 114239 T306 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4:junit4]   2> 114240 T306 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index/
[junit4:junit4]   2> 114240 T306 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 114241 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 114242 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 114242 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 114243 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 114243 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 114244 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 114244 T306 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 114244 T306 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 114245 T306 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 114245 T306 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 114246 T306 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 114246 T306 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 114246 T306 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 114247 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 114247 T306 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 114248 T306 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 114248 T306 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114249 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114249 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114250 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114250 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114251 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114251 T306 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114252 T306 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 114252 T306 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 114253 T306 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 114255 T306 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 114256 T306 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 114257 T306 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [SOLRCONFIG-MANAGED-SCHEMA.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SOLRCONFIG-MANAGED-SCHEMA-NAMED-SCHEMA.XML.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SCHEMA-BEHAVIOR.XML, SOLRCONFIG-MERGEPOLICY-DEFAULTS.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, STOP-2.TXT, SCHEMA11.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, SCHEMA15.XML, BAD-SOLRCONFIG-UNEXPECTED-SCHEMA-ATTRIBUTE.XML, SOLRCONFIG-NATIVELOCK.XML, SCHEMA-SIM.XML, CONDITIONAL.UPDATEPROCESSOR.JS, SCHEMA-DOCVALUES.XML, SCHEMA-ID-AND-VERSION-FIELDS-ONLY.XML, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-DOCVALUES-NOT-REQUIRED-NO-DEFAULT.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, SCHEMA-EFF.XML, SOLRCONFIG_CODEC.XML, SOLRCONFIG-MUTABLE-MANAGED-SCHEMA.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML, HYPHENATION.DTD, TRIVIAL.UPDATEPROCESSOR1.JS, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-PHRASESUGGEST.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-BASIC.XML, SCHEMA-LMJELINEKMERCER.XML, BAD-MP-SOLRCONFIG.XML, JASUGGEST.TXT, BAD-SCHEMA-DUP-FIELD.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SOLRCONFIG-MASTER.XML, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-TFIDF.XML, SCHEMA-REPLICATION1.XML, STOPWITHBOM.TXT, SOLRCONFIG-ELEVATE.XML, SYNONYMS.TXT, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-COMPONENTS-NAME.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, WDFTYPES.TXT, BAD-SCHEMA-CURRENCY-FT-OER-NORATES.XML, SOLRCONFIG-DELPOLICY1.XML, BAD_SOLRCONFIG.XML, BAD-SCHEMA-UNSUPPORTED-DOCVALUES.XML, HUNSPELL-TEST.DIC, SCHEMA-NUMERIC.XML, SCHEMA-LUCENEMATCHVERSION.XML, KEEP-2.TXT, SCHEMA-STOP-KEEP.XML, SCHEMA-IB.XML, STOP-SNOWBALL.TXT, SCHEMA-ONE-FIELD-NO-DYNAMIC-FIELD.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-DOCVALUESMULTI.XML, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG-SLAVE.XML, SCHEMA-CLASS-NAME-SHORTENING-ON-SERIALIZATION.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, STOPWORDSWRONGENCODING.TXT, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, STOPTYPES-1.TXT, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SOLRCONFIG.SNIPPET.RANDOMINDEXCONFIG.XML, REGEX-BOOST-PROCESSOR-TEST.TXT, SOLRCONFIG-MINIMAL.XML, SCHEMA-DOCVALUESFACETING.XML, SOLRCONFIG-TLOG.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-POSTINGSHIGHLIGHT.XML, DA_COMPOUNDDICTIONARY.TXT, SOLRCONFIG-TERMINDEX.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, BAD-SCHEMA-NON-GLOB-COPYFIELD-SOURCE-MATCHING-NOTHING-SHOULD-FAIL-TEST.XML, SCHEMASURROUND.XML, OPEN-EXCHANGE-RATES.JSON, BAD-SCHEMA-DUP-FIELDTYPE.XML, PHRASESUGGEST.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-MASTER2.XML, OLD_SYNONYMS.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SCHEMA-LMDIRICHLET.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-FOLDING.XML, SOLRCONFIG-CACHING.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD-CURRENCY.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SCHEMA-BM25.XML, SOLRCONFIG-ENABLEPLUGIN.XML, STOP-1.TXT, BAD-SCHEMA-SWEETSPOT-BOTH-TF.XML, SCHEMA-REST.XML, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SCHEMA12.XML, SOLRCONFIG-TRANSFORMERS.XML, SCHEMA-COPYFIELD-TEST.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-BASELINE.XML, CURRENCY.XML, DA_UTF8.XML, FUZZYSUGGEST.TXT, SCHEMA-POSTINGSHIGHLIGHT.XML, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SOLRCONFIG-SOLR-749.XML, STOPWORDS.TXT, BAD-SCHEMA-BOGUS-ANALYSIS-PARAMETERS.XML, HUNSPELL-TEST.AFF, SOLRCONFIG-TLOG-MANAGED-SCHEMA.XML, SOLRCONFIG_PERF.XML, SCHEMA-SWEETSPOT.XML, SOLRCONFIG-SIMPLELOCK.XML, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-TINY.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-NORMS.XML, FRENCHARTICLES.TXT, SOLRCONFIG-LAZYWRITER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SCHEMA-REPLICATION2.XML, SCHEMA-SNIPPET-TYPE.XML, SCHEMA-SPATIAL.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-HYPERBOLIC.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-XINCLUDE.XML, XSLT, SOLRCONFIG-WARMER.XML, SOLRCONFIG-DELPOLICY2.XML, KEEP-1.TXT, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA-TRIE.XML, SOLRCONFIG-TLOG-MUTABLE-MANAGED-SCHEMA.XML, SCHEMA-BINARYFIELD.XML, SCHEMA-REST-LUCENE-MATCH-VERSION.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-CODE-IN-XML.XML, SCHEMA-MINIMAL.XML, STOPTYPES-2.TXT, SCHEMA_CODEC.XML, STEMDICT.TXT, SCHEMA-SYNONYM-TOKENIZER.XML, ELEVATE.XML, SCHEMA-COLLATE.XML, PROTWORDS.TXT, SOLRCONFIG-DEFAULTS.XML, SCHEMA-XINCLUDE.XML, SCHEMA-SNIPPET-FIELD.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMA-PHRASESUGGEST.XML, SCHEMA-DFR.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SOLRCONFIG-MASTER1.XML, SOLRCONFIG-FUNCTIONQUERY.XML, BAD-SOLRCONFIG-SCHEMA-MUTABLE-BUT-NOT-MANAGED.XML, SOLRCONFIG-MASTER3.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, SOLRCONFIG-ALTDIRECTORY.XML, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-DEFAULT-CODE.XML, SOLRCONFIG-MERGEPOLICY-LEGACY.XML, BAD-SCHEMA-BOGUS-FIELD-PARAMETERS.XML]
[junit4:junit4]   2> 114259 T306 oass.SolrIndexSearcher.<init> Opening Searcher@aa9b1c1c main
[junit4:junit4]   2> 114260 T306 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 114260 T306 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 114261 T306 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 114262 T306 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 114268 T306 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 114268 T310 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@aa9b1c1c main{StandardDirectoryReader(segments_1:5:nrt)}
[junit4:junit4]   2> 114268 T306 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 114269 T306 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4]   2> 114270 T306 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_2,generation=2}
[junit4:junit4]   2> 114270 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 114322 T306 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_2,generation=2}
[junit4:junit4]   2> 114323 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 114323 T306 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 114323 T306 oass.SolrIndexSearcher.<init> Opening Searcher@6f1c5098 main
[junit4:junit4]   2> 114324 T306 oasc.CoreContainer.registerCore replacing core: collection1
[junit4:junit4]   2> 114324 T306 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4cf9e62
[junit4:junit4]   2> 114324 T310 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f1c5098 main{StandardDirectoryReader(segments_2:5:nrt)}
[junit4:junit4]   2> 114325 T306 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommit maxTime=1000ms,soft autocommits=1,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 114326 T306 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 114356 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[0 (1438878912604012544)]} 0 28
[junit4:junit4]   2> 114357 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[1 (1438878912607158272)]} 0 28
[junit4:junit4]   2> 114358 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438878912608206848)]} 0 28
[junit4:junit4]   2> 114359 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[3 (1438878912609255424)]} 0 28
[junit4:junit4]   2> 114361 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[4 (1438878912610304000)]} 0 28
[junit4:junit4]   2> 114362 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[5 (1438878912611352576)]} 0 28
[junit4:junit4]   2> 114363 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[6 (1438878912613449728)]} 0 28
[junit4:junit4]   2> 114364 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[7 (1438878912614498304)]} 0 28
[junit4:junit4]   2> 114365 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[8 (1438878912615546880)]} 0 28
[junit4:junit4]   2> 114370 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[9 (1438878912616595456)]} 0 28
[junit4:junit4]   2> 114371 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[10 (1438878912620789760)]} 0 28
[junit4:junit4]   2> 114372 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[11 (1438878912622886912)]} 0 28
[junit4:junit4]   2> 114374 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[12 (1438878912623935488)]} 0 28
[junit4:junit4]   2> 114375 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[13 (1438878912624984064)]} 0 28
[junit4:junit4]   2> ASYNC  NEW_CORE C415 name=collection1 org.apache.solr.core.SolrCore@7e014e61
[junit4:junit4]   2> 114635 T306 C415 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:1&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=260 
[junit4:junit4]   2> 114666 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[14 (1438878912930119680)]} 0 28
[junit4:junit4]   2> 114667 T311 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4]   2> 114670 T311 oass.SolrIndexSearcher.<init> Opening Searcher@16932868 main
[junit4:junit4]   2> 114671 T311 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 114671 T310 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16932868 main{StandardDirectoryReader(segments_2:9:nrt _1(5.0):C10 _2(5.0):C5)}
[junit4:junit4]   2> 114918 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[15 (1438878913194360832)]} 0 28
[junit4:junit4]   2> 114919 T306 C415 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:14&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 114920 T306 C415 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:15&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 114921 T306 oas.SolrTestCaseJ4.tearDown ###Ending testMaxDocs
[junit4:junit4]   2> 114926 T306 oas.SolrTestCaseJ4.setUp ###Starting testCommitWithin
[junit4:junit4]   2> ASYNC  NEW_CORE C416 name=collection1 org.apache.solr.core.SolrCore@7e014e61
[junit4:junit4]   2> 114942 T306 C416 oass.SolrIndexSearcher.<init> Opening Searcher@706b777c realtime
[junit4:junit4]   2> 114942 T306 C416 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1438878913209040896)} 0 11
[junit4:junit4]   2> 114944 T306 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 114944 T306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 114945 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 114946 T306 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 114975 T306 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 115018 T306 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 115018 T306 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 115023 T306 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 115457 T306 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 115464 T306 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 115468 T306 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 115473 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 115477 T306 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 115480 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 115480 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 115481 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 115481 T306 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 115482 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 115482 T306 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 115483 T306 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/
[junit4:junit4]   2> 115483 T306 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4:junit4]   2> 115484 T306 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index/
[junit4:junit4]   2> 115484 T306 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 115485 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 115485 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 115486 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 115486 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 115487 T306 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 115487 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 115488 T306 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 115488 T306 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 115489 T306 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 115489 T306 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 115489 T306 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 115490 T306 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 115490 T306 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 115491 T306 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 115491 T306 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 115491 T306 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 115492 T306 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115492 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115492 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115493 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115493 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115494 T306 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115494 T306 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115494 T306 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 115495 T306 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 115496 T306 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 115497 T306 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 115498 T306 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 115500 T306 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [SOLRCONFIG-MANAGED-SCHEMA.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SOLRCONFIG-MANAGED-SCHEMA-NAMED-SCHEMA.XML.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SCHEMA-BEHAVIOR.XML, SOLRCONFIG-MERGEPOLICY-DEFAULTS.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, STOP-2.TXT, SCHEMA11.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, SCHEMA15.XML, BAD-SOLRCONFIG-UNEXPECTED-SCHEMA-ATTRIBUTE.XML, SOLRCONFIG-NATIVELOCK.XML, SCHEMA-SIM.XML, CONDITIONAL.UPDATEPROCESSOR.JS, SCHEMA-DOCVALUES.XML, SCHEMA-ID-AND-VERSION-FIELDS-ONLY.XML, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-DOCVALUES-NOT-REQUIRED-NO-DEFAULT.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, SCHEMA-EFF.XML, SOLRCONFIG_CODEC.XML, SOLRCONFIG-MUTABLE-MANAGED-SCHEMA.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML, HYPHENATION.DTD, TRIVIAL.UPDATEPROCESSOR1.JS, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-PHRASESUGGEST.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-BASIC.XML, SCHEMA-LMJELINEKMERCER.XML, BAD-MP-SOLRCONFIG.XML, JASUGGEST.TXT, BAD-SCHEMA-DUP-FIELD.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SOLRCONFIG-MASTER.XML, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-TFIDF.XML, SCHEMA-REPLICATION1.XML, STOPWITHBOM.TXT, SOLRCONFIG-ELEVATE.XML, SYNONYMS.TXT, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-COMPONENTS-NAME.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, WDFTYPES.TXT, BAD-SCHEMA-CURRENCY-FT-OER-NORATES.XML, SOLRCONFIG-DELPOLICY1.XML, BAD_SOLRCONFIG.XML, BAD-SCHEMA-UNSUPPORTED-DOCVALUES.XML, HUNSPELL-TEST.DIC, SCHEMA-NUMERIC.XML, SCHEMA-LUCENEMATCHVERSION.XML, KEEP-2.TXT, SCHEMA-STOP-KEEP.XML, SCHEMA-IB.XML, STOP-SNOWBALL.TXT, SCHEMA-ONE-FIELD-NO-DYNAMIC-FIELD.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-DOCVALUESMULTI.XML, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG-SLAVE.XML, SCHEMA-CLASS-NAME-SHORTENING-ON-SERIALIZATION.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, STOPWORDSWRONGENCODING.TXT, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, STOPTYPES-1.TXT, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SOLRCONFIG.SNIPPET.RANDOMINDEXCONFIG.XML, REGEX-BOOST-PROCESSOR-TEST.TXT, SOLRCONFIG-MINIMAL.XML, SCHEMA-DOCVALUESFACETING.XML, SOLRCONFIG-TLOG.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-POSTINGSHIGHLIGHT.XML, DA_COMPOUNDDICTIONARY.TXT, SOLRCONFIG-TERMINDEX.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, BAD-SCHEMA-NON-GLOB-COPYFIELD-SOURCE-MATCHING-NOTHING-SHOULD-FAIL-TEST.XML, SCHEMASURROUND.XML, OPEN-EXCHANGE-RATES.JSON, BAD-SCHEMA-DUP-FIELDTYPE.XML, PHRASESUGGEST.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-MASTER2.XML, OLD_SYNONYMS.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SCHEMA-LMDIRICHLET.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-FOLDING.XML, SOLRCONFIG-CACHING.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD-CURRENCY.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SCHEMA-BM25.XML, SOLRCONFIG-ENABLEPLUGIN.XML, STOP-1.TXT, BAD-SCHEMA-SWEETSPOT-BOTH-TF.XML, SCHEMA-REST.XML, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SCHEMA12.XML, SOLRCONFIG-TRANSFORMERS.XML, SCHEMA-COPYFIELD-TEST.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-BASELINE.XML, CURRENCY.XML, DA_UTF8.XML, FUZZYSUGGEST.TXT, SCHEMA-POSTINGSHIGHLIGHT.XML, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SOLRCONFIG-SOLR-749.XML, STOPWORDS.TXT, BAD-SCHEMA-BOGUS-ANALYSIS-PARAMETERS.XML, HUNSPELL-TEST.AFF, SOLRCONFIG-TLOG-MANAGED-SCHEMA.XML, SOLRCONFIG_PERF.XML, SCHEMA-SWEETSPOT.XML, SOLRCONFIG-SIMPLELOCK.XML, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-TINY.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-NORMS.XML, FRENCHARTICLES.TXT, SOLRCONFIG-LAZYWRITER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SCHEMA-REPLICATION2.XML, SCHEMA-SNIPPET-TYPE.XML, SCHEMA-SPATIAL.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-HYPERBOLIC.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-XINCLUDE.XML, XSLT, SOLRCONFIG-WARMER.XML, SOLRCONFIG-DELPOLICY2.XML, KEEP-1.TXT, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA-TRIE.XML, SOLRCONFIG-TLOG-MUTABLE-MANAGED-SCHEMA.XML, SCHEMA-BINARYFIELD.XML, SCHEMA-REST-LUCENE-MATCH-VERSION.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-CODE-IN-XML.XML, SCHEMA-MINIMAL.XML, STOPTYPES-2.TXT, SCHEMA_CODEC.XML, STEMDICT.TXT, SCHEMA-SYNONYM-TOKENIZER.XML, ELEVATE.XML, SCHEMA-COLLATE.XML, PROTWORDS.TXT, SOLRCONFIG-DEFAULTS.XML, SCHEMA-XINCLUDE.XML, SCHEMA-SNIPPET-FIELD.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMA-PHRASESUGGEST.XML, SCHEMA-DFR.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SOLRCONFIG-MASTER1.XML, SOLRCONFIG-FUNCTIONQUERY.XML, BAD-SOLRCONFIG-SCHEMA-MUTABLE-BUT-NOT-MANAGED.XML, SOLRCONFIG-MASTER3.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, SOLRCONFIG-ALTDIRECTORY.XML, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-DEFAULT-CODE.XML, SOLRCONFIG-MERGEPOLICY-LEGACY.XML, BAD-SCHEMA-BOGUS-FIELD-PARAMETERS.XML]
[junit4:junit4]   2> 115501 T306 oass.SolrIndexSearcher.<init> Opening Searcher@180313b5 main
[junit4:junit4]   2> 115502 T306 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 115502 T306 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 115503 T306 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 115504 T306 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 115512 T312 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@180313b5 main{StandardDirectoryReader(segments_2:13:nrt)}
[junit4:junit4]   2> 115512 T306 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 115513 T306 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 115513 T306 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4]   2> 115515 T306 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_3,generation=3}
[junit4:junit4]   2> 115515 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 115519 T306 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_3,generation=3}
[junit4:junit4]   2> 115519 T306 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 115520 T306 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 115521 T306 oass.SolrIndexSearcher.<init> Opening Searcher@646e1493 main
[junit4:junit4]   2> 115522 T306 oasc.CoreContainer.registerCore replacing core: collection1
[junit4:junit4]   2> 115522 T306 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7e014e61
[junit4:junit4]   2> 115522 T312 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@646e1493 main{StandardDirectoryReader(segments_3:13:nrt)}
[junit4:junit4]   2> 115523 T306 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommit maxDocs=14,soft autocommits=1,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=16,adds=1,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=16,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 115524 T306 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 115529 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[529 (1438878913833992192)]} 0 2
[junit4:junit4]   2> ASYNC  NEW_CORE C417 name=collection1 org.apache.solr.core.SolrCore@b4bcd1bf
[junit4:junit4]   2> 115530 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:529&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 117529 T313 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4]   2> 117532 T313 oass.SolrIndexSearcher.<init> Opening Searcher@e21464a0 main
[junit4:junit4]   2> 117532 T313 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 117532 T312 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e21464a0 main{StandardDirectoryReader(segments_3:15:nrt _4(5.0):C1)}
[junit4:junit4]   2> 117783 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[530 (1438878916198531072)]} 0 2
[junit4:junit4]   2> 117784 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 117785 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[529 (-1438878916201676800)]} 0 2
[junit4:junit4]   2> 117786 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:529&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 118785 T313 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4]   2> 118789 T313 oass.SolrIndexSearcher.<init> Opening Searcher@8b070652 main
[junit4:junit4]   2> 118789 T313 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 118790 T312 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8b070652 main{StandardDirectoryReader(segments_3:19:nrt _5(5.0):C1)}
[junit4:junit4]   2> 119038 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:529&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 119039 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 119041 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917518688256)]} 0 2
[junit4:junit4]   2> 119042 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917519736832)]} 0 2
[junit4:junit4]   2> 119043 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917520785408)]} 0 2
[junit4:junit4]   2> 119044 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917520785409)]} 0 2
[junit4:junit4]   2> 119044 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917521833984)]} 0 2
[junit4:junit4]   2> 119045 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917522882560)]} 0 2
[junit4:junit4]   2> 119046 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917523931136)]} 0 2
[junit4:junit4]   2> 119047 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917523931137)]} 0 2
[junit4:junit4]   2> 119047 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917524979712)]} 0 2
[junit4:junit4]   2> 119050 T306 oasup.LogUpdateProcessor.finish [collection1] {add=[500 (1438878917526028288)]} 0 2
[junit4:junit4]   2> 119051 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:500&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 119053 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917531271168)]} 0 2
[junit4:junit4]   2> 119053 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917531271169)]} 0 2
[junit4:junit4]   2> 119054 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917532319744)]} 0 2
[junit4:junit4]   2> 119054 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917532319745)]} 0 2
[junit4:junit4]   2> 119055 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917533368320)]} 0 2
[junit4:junit4]   2> 119055 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917533368321)]} 0 2
[junit4:junit4]   2> 119055 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917533368322)]} 0 2
[junit4:junit4]   2> 119056 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917534416896)]} 0 2
[junit4:junit4]   2> 119056 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917534416897)]} 0 2
[junit4:junit4]   2> 119057 T306 oasup.LogUpdateProcessor.finish [collection1] {delete=[530 (-1438878917535465472)]} 0 2
[junit4:junit4]   2> 119057 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 120053 T313 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4:junit4]   2> 120055 T313 oass.SolrIndexSearcher.<init> Opening Searcher@b3ce24d9 main
[junit4:junit4]   2> 120055 T313 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 120055 T312 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b3ce24d9 main{StandardDirectoryReader(segments_3:23:nrt _6(5.0):C10/9)}
[junit4:junit4]   2> 120059 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:500&start=0&qt=standard&version=2.2&rows=20} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 120061 T306 C417 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:530&start=0&qt=standard&version=2.2&rows=20} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 120062 T306 oas.SolrTestCaseJ4.tearDown ###Ending testCommitWithin
[junit4:junit4]   2> 120064 T306 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 120064 T306 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-758671123
[junit4:junit4]   2> 120065 T306 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@b4bcd1bf
[junit4:junit4]   2> 120065 T306 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=3,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=12,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=12,cumulative_deletesById=11,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 120065 T306 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 120066 T306 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 120066 T306 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 120067 T306 C417 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_3,generation=3}
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@dc201e28 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b82208e)),segFN=segments_4,generation=4}
[junit4:junit4]   2> 120067 T306 C417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4]   2> 120068 T306 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 120068 T306 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   2> 120068 T306 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index;done=false>>]
[junit4:junit4]   2> 120069 T306 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568/index
[junit4:junit4]   2> 120069 T306 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568;done=false>>]
[junit4:junit4]   2> 120069 T306 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-AutoCommitTest-1372221858568
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=lv_LV_PREEURO, timezone=Atlantic/Jan_Mayen
[junit4:junit4]   2> NOTE: Linux 3.2.0-48-generic amd64/IBM Corporation 1.7.0 (64-bit)/cpus=8,threads=1,free=22927880,total=116785152
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPropInjectDefaults, ExternalFileFieldSortTest, DistributedSpellCheckComponentTest, TestPseudoReturnFields, TestStressRecovery, TestFaceting, TestSort, TestDefaultSearchFieldResource, ShardRoutingTest, TestSurroundQueryParser, TestDynamicFieldCollectionResource, TestWriterPerf, SpatialFilterTest, RegexBoostProcessorTest, SuggesterFSTTest, MBeansHandlerTest, TestManagedSchemaFieldResource, SolrTestCaseJ4Test, QueryElevationComponentTest, JsonLoaderTest, IndexSchemaTest, TestPhraseSuggestions, SpellingQueryConverterTest, TermVectorComponentTest, TestXIncludeConfig, DirectSolrConnectionTest, TestCoreContainer, TestFieldCollectionResource, SchemaVersionSpecificBehaviorTest, SystemInfoHandlerTest, PolyFieldTest, SOLR749Test, BadCopyFieldTest, AutoCommitTest]
[junit4:junit4] Completed on J0 in 10.54s, 3 tests, 1 error <<< FAILURES!

[...truncated 694 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:379: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1243: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 error, 36 ignored (7 assumptions)

Total time: 29 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure