You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/19 04:14:22 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1342 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1342/
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 600 from http://127.0.0.1:52407/collection1lastClient and got 598 from http://127.0.0.1:52425/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 600 from http://127.0.0.1:52407/collection1lastClient and got 598 from http://127.0.0.1:52425/collection1
at __randomizedtesting.SeedInfo.seed([7093929FF69C3DF4:F1751C8781C35DC8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:64185 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:64185 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([7093929FF69C3DF4:F1751C8781C35DC8]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9306 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4] 2> 10145 T27 oas.SolrTestCaseJ4.startTrackingSearchers WARN startTrackingSearchers: numOpens=1 numCloses=1
[junit4:junit4] 2> 10226 T27 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 10259 T27 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1371606033768
[junit4:junit4] 2> 10268 T27 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 10275 T28 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 10476 T27 oasc.ZkTestServer.run start zk server on port:60574
[junit4:junit4] 2> 10782 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 10939 T34 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@192a36a0 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 10940 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 10947 T27 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 11076 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 11080 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@400a2506 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 11081 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 11099 T27 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 11108 T27 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 11115 T27 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 11122 T27 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 11130 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 11167 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 11185 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 11186 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 11295 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 11297 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 11305 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 11306 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 11314 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 11315 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 11323 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 11324 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 11331 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 11333 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 11341 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 11342 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 11349 T27 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 11351 T27 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 11693 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 11698 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63806
[junit4:junit4] 2> 11700 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 11701 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 11702 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868
[junit4:junit4] 2> 11703 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/solr.xml
[junit4:junit4] 2> 11704 T27 oasc.CoreContainer.<init> New CoreContainer 97033092
[junit4:junit4] 2> 11705 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/'
[junit4:junit4] 2> 11706 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/'
[junit4:junit4] 2> 12055 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12057 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12058 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12059 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12060 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12061 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12062 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12062 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12063 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12064 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12091 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 12093 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 12144 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12146 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12152 T47 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d3d888e name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12153 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12166 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12182 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12185 T49 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54d9a624 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12185 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12205 T27 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 12229 T27 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 12260 T27 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 12265 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63806_
[junit4:junit4] 2> 12277 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63806_
[junit4:junit4] 2> 12284 T27 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 12327 T27 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 12332 T27 oasc.Overseer.start Overseer (id=89889573047894019-127.0.0.1:63806_-n_0000000000) starting
[junit4:junit4] 2> 12379 T27 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 12406 T51 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 12406 T27 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 12412 T27 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 12415 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 12452 T50 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 12465 T52 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 12466 T52 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13964 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13967 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63806_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63806"}
[junit4:junit4] 2> 13967 T50 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 13998 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 14033 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 14474 T52 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/collection1
[junit4:junit4] 2> 14475 T52 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 14476 T52 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 14477 T52 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 14481 T52 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/collection1/'
[junit4:junit4] 2> 14483 T52 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 14485 T52 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/collection1/lib/README' to classloader
[junit4:junit4] 2> 14664 T52 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 14809 T52 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 14912 T52 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 15015 T52 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 17300 T52 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17349 T52 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17353 T52 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 18282 T52 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 18300 T52 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 18306 T52 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 18312 T52 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 18312 T52 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 18313 T52 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 18315 T52 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 18315 T52 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 18316 T52 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 18347 T52 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371606034868/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data/
[junit4:junit4] 2> 18355 T52 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 18360 T52 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 18397 T52 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data
[junit4:junit4] 2> 18398 T52 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data/index/
[junit4:junit4] 2> 18399 T52 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 18400 T52 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data/index
[junit4:junit4] 2> 18419 T52 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 18420 T52 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 18524 T52 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 18524 T52 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 18637 T52 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 18643 T52 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 18664 T52 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 18664 T52 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 18665 T52 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 18666 T52 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 18671 T52 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 18682 T52 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 18691 T52 oass.SolrIndexSearcher.<init> Opening Searcher@589e0cdc main
[junit4:junit4] 2> 18716 T52 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 18717 T52 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 18787 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@589e0cdc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 18792 T52 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 18793 T52 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63806 collection:control_collection shard:shard1
[junit4:junit4] 2> 18805 T52 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 18820 T52 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 18838 T52 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 18851 T52 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 18851 T52 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 18855 T52 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63806/collection1/
[junit4:junit4] 2> 18856 T52 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 18856 T52 oasc.SyncStrategy.syncToMe http://127.0.0.1:63806/collection1/ has no replicas
[junit4:junit4] 2> 18857 T52 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63806/collection1/
[junit4:junit4] 2> 18857 T52 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 20051 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20079 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 20143 T52 oasc.ZkController.register We are http://127.0.0.1:63806/collection1/ and leader is http://127.0.0.1:63806/collection1/
[junit4:junit4] 2> 20149 T52 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63806
[junit4:junit4] 2> 20149 T52 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 20150 T52 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20151 T52 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20154 T52 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20157 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 20158 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 20159 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 20175 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 20183 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20186 T56 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52709cc name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 20187 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 20190 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 20197 T27 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 20479 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 20483 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39436
[junit4:junit4] 2> 20484 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 20485 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 20486 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706
[junit4:junit4] 2> 20487 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/solr.xml
[junit4:junit4] 2> 20488 T27 oasc.CoreContainer.<init> New CoreContainer 2007523556
[junit4:junit4] 2> 20489 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/'
[junit4:junit4] 2> 20489 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/'
[junit4:junit4] 2> 20653 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 20654 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 20655 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 20656 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 20657 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 20658 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 20658 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 20659 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 20660 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 20661 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 20682 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 20683 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 20684 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 20685 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20700 T67 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@439b2c10 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 20702 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 20716 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 20731 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20734 T69 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46c4ee90 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 20735 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 20743 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 21598 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21601 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63806__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63806_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63806"}
[junit4:junit4] 2> 21621 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 21621 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 21621 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 21749 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39436_
[junit4:junit4] 2> 21752 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39436_
[junit4:junit4] 2> 21756 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 21756 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 21757 T69 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 21758 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 21759 T56 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 21761 T49 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 21774 T70 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 21774 T70 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 23129 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23131 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39436_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39436"}
[junit4:junit4] 2> 23132 T50 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4] 2> 23133 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 23151 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 23151 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 23151 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 23779 T70 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/collection1
[junit4:junit4] 2> 23780 T70 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 23781 T70 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 23782 T70 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 23785 T70 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/collection1/'
[junit4:junit4] 2> 23787 T70 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 23788 T70 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/collection1/lib/README' to classloader
[junit4:junit4] 2> 23863 T70 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 23961 T70 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 24063 T70 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 24140 T70 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 24965 T70 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 24989 T70 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 24994 T70 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 25043 T70 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 25052 T70 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 25060 T70 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 25063 T70 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 25063 T70 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 25064 T70 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 25066 T70 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 25067 T70 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 25068 T70 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 25068 T70 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371606043706/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/
[junit4:junit4] 2> 25069 T70 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 25071 T70 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 25072 T70 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1
[junit4:junit4] 2> 25074 T70 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index/
[junit4:junit4] 2> 25075 T70 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 25076 T70 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index
[junit4:junit4] 2> 25084 T70 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 25085 T70 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 25090 T70 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 25091 T70 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 25093 T70 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 25096 T70 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 25099 T70 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 25100 T70 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 25101 T70 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 25104 T70 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 25105 T70 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 25122 T70 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 25133 T70 oass.SolrIndexSearcher.<init> Opening Searcher@4e4fc091 main
[junit4:junit4] 2> 25135 T70 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 25135 T70 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 25163 T71 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e4fc091 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 25172 T70 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 25173 T70 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39436 collection:collection1 shard:shard1
[junit4:junit4] 2> 25175 T70 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 25195 T70 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 25207 T70 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 25207 T70 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 25208 T70 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39436/collection1/
[junit4:junit4] 2> 25208 T70 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 25209 T70 oasc.SyncStrategy.syncToMe http://127.0.0.1:39436/collection1/ has no replicas
[junit4:junit4] 2> 25209 T70 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39436/collection1/
[junit4:junit4] 2> 25210 T70 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 26163 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 26185 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26185 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26185 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26190 T70 oasc.ZkController.register We are http://127.0.0.1:39436/collection1/ and leader is http://127.0.0.1:39436/collection1/
[junit4:junit4] 2> 26190 T70 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39436
[junit4:junit4] 2> 26191 T70 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 26191 T70 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 26192 T70 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 26198 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 26201 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 26202 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 26202 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 26498 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 26502 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45842
[junit4:junit4] 2> 26503 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 26504 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 26505 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729
[junit4:junit4] 2> 26506 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/solr.xml
[junit4:junit4] 2> 26506 T27 oasc.CoreContainer.<init> New CoreContainer 364231735
[junit4:junit4] 2> 26507 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/'
[junit4:junit4] 2> 26508 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/'
[junit4:junit4] 2> 26672 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 26673 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 26674 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 26675 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 26676 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 26676 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 26677 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 26678 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 26679 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 26680 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 26698 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 26699 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 26700 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 26701 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 26720 T83 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24657bef name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 26721 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 26725 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 26739 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 26742 T85 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78b2fe03 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 26743 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 26756 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 27702 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 27704 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:39436__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39436_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39436"}
[junit4:junit4] 2> 27722 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27722 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27722 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27722 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27762 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45842_
[junit4:junit4] 2> 27765 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45842_
[junit4:junit4] 2> 27769 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27769 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27770 T85 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 27770 T69 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 27771 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 27771 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 27772 T56 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 27774 T49 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 27781 T86 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 27782 T86 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 29231 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 29233 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45842_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45842"}
[junit4:junit4] 2> 29234 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 29235 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 29252 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 29252 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 29252 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 29252 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 29786 T86 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/collection1
[junit4:junit4] 2> 29787 T86 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 29788 T86 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 29789 T86 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 29792 T86 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/collection1/'
[junit4:junit4] 2> 29794 T86 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 29795 T86 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/collection1/lib/README' to classloader
[junit4:junit4] 2> 29866 T86 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 29946 T86 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 30049 T86 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 30117 T86 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 30793 T86 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 30809 T86 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 30812 T86 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 30846 T86 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 30852 T86 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 30858 T86 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 30860 T86 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 30860 T86 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 30861 T86 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 30863 T86 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 30863 T86 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 30863 T86 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 30864 T86 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371606049729/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/
[junit4:junit4] 2> 30864 T86 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 30865 T86 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 30866 T86 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2
[junit4:junit4] 2> 30867 T86 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index/
[junit4:junit4] 2> 30868 T86 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 30869 T86 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index
[junit4:junit4] 2> 30874 T86 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 30874 T86 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 30878 T86 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 30878 T86 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 30879 T86 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 30881 T86 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 30883 T86 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 30883 T86 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 30884 T86 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 30885 T86 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 30886 T86 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 30897 T86 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 30904 T86 oass.SolrIndexSearcher.<init> Opening Searcher@1f2e99f0 main
[junit4:junit4] 2> 30905 T86 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 30905 T86 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 30921 T87 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f2e99f0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 30928 T86 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 30928 T86 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45842 collection:collection1 shard:shard2
[junit4:junit4] 2> 30929 T86 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 30947 T86 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 30958 T86 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 30959 T86 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 30959 T86 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45842/collection1/
[junit4:junit4] 2> 30959 T86 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 30960 T86 oasc.SyncStrategy.syncToMe http://127.0.0.1:45842/collection1/ has no replicas
[junit4:junit4] 2> 30960 T86 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45842/collection1/
[junit4:junit4] 2> 30960 T86 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 32264 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 32286 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 32286 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 32286 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 32286 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 32302 T86 oasc.ZkController.register We are http://127.0.0.1:45842/collection1/ and leader is http://127.0.0.1:45842/collection1/
[junit4:junit4] 2> 32302 T86 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45842
[junit4:junit4] 2> 32303 T86 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 32303 T86 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 32304 T86 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 32307 T86 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 32309 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 32310 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 32311 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 32599 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 32603 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:53771
[junit4:junit4] 2> 32604 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 32605 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 32606 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838
[junit4:junit4] 2> 32606 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/solr.xml
[junit4:junit4] 2> 32607 T27 oasc.CoreContainer.<init> New CoreContainer 1446852057
[junit4:junit4] 2> 32608 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/'
[junit4:junit4] 2> 32609 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/'
[junit4:junit4] 2> 32756 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 32757 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 32758 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 32758 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 32759 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 32760 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 32761 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 32761 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 32762 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 32763 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 32781 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 32782 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 32783 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 32784 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 32788 T99 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27045171 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 32788 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 32792 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 32805 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 32808 T101 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4480e1f8 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 32809 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 32821 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 33802 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 33804 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:45842__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45842_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45842"}
[junit4:junit4] 2> 33822 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33822 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33822 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33822 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33822 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33827 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53771_
[junit4:junit4] 2> 33830 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53771_
[junit4:junit4] 2> 33834 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33834 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 33835 T101 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 33835 T69 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 33835 T85 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 33836 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 33836 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 33837 T56 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 33837 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 33839 T49 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 33846 T102 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 33847 T102 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 35330 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 35333 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53771_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53771"}
[junit4:junit4] 2> 35333 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 35334 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 35354 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 35355 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 35355 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 35355 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 35354 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 35851 T102 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/collection1
[junit4:junit4] 2> 35852 T102 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 35853 T102 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 35854 T102 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 35857 T102 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/collection1/'
[junit4:junit4] 2> 35859 T102 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 35860 T102 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/collection1/lib/README' to classloader
[junit4:junit4] 2> 35930 T102 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 36007 T102 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 36109 T102 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 36173 T102 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 37012 T102 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 37038 T102 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 37043 T102 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 37078 T102 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 37084 T102 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 37090 T102 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 37092 T102 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 37092 T102 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 37093 T102 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 37095 T102 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 37095 T102 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 37095 T102 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 37096 T102 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371606055838/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3/
[junit4:junit4] 2> 37096 T102 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 37097 T102 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 37099 T102 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3
[junit4:junit4] 2> 37100 T102 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3/index/
[junit4:junit4] 2> 37101 T102 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 37102 T102 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3/index
[junit4:junit4] 2> 37108 T102 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 37108 T102 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 37113 T102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 37113 T102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 37115 T102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 37116 T102 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 37117 T102 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 37117 T102 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 37118 T102 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 37119 T102 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 37120 T102 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 37139 T102 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 37149 T102 oass.SolrIndexSearcher.<init> Opening Searcher@25137b50 main
[junit4:junit4] 2> 37151 T102 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 37151 T102 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 37161 T103 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25137b50 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 37166 T102 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 37166 T102 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53771 collection:collection1 shard:shard3
[junit4:junit4] 2> 37167 T102 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4] 2> 37184 T102 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 37194 T102 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 37195 T102 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 37195 T102 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53771/collection1/
[junit4:junit4] 2> 37196 T102 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 37196 T102 oasc.SyncStrategy.syncToMe http://127.0.0.1:53771/collection1/ has no replicas
[junit4:junit4] 2> 37196 T102 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53771/collection1/
[junit4:junit4] 2> 37197 T102 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4] 2> 38366 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 38387 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 38387 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 38387 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 38387 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 38387 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 38437 T102 oasc.ZkController.register We are http://127.0.0.1:53771/collection1/ and leader is http://127.0.0.1:53771/collection1/
[junit4:junit4] 2> 38437 T102 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53771
[junit4:junit4] 2> 38437 T102 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 38438 T102 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 38438 T102 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 38441 T102 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 38443 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 38444 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 38445 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 38736 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 38740 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63848
[junit4:junit4] 2> 38741 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 38742 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 38743 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972
[junit4:junit4] 2> 38743 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/solr.xml
[junit4:junit4] 2> 38744 T27 oasc.CoreContainer.<init> New CoreContainer 840420812
[junit4:junit4] 2> 38745 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/'
[junit4:junit4] 2> 38746 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/'
[junit4:junit4] 2> 38892 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 38893 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 38893 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 38894 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 38895 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 38895 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 38896 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 38897 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 38898 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 38898 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 38919 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 38920 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 38920 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 38922 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 38936 T115 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3569158f name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 38937 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 38947 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 38961 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 38964 T117 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f931514 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 38964 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 38977 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 39902 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 39904 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:53771__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53771_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53771"}
[junit4:junit4] 2> 39973 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39973 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39973 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39973 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39973 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39973 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39983 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63848_
[junit4:junit4] 2> 39986 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63848_
[junit4:junit4] 2> 39990 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39990 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39991 T69 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 39990 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 39992 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 39991 T101 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 39991 T85 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 39994 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 39994 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 40000 T117 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 40001 T49 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 40002 T56 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 40009 T118 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 40010 T118 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 41482 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 41484 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63848"}
[junit4:junit4] 2> 41484 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 41485 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 41498 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 41498 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 41498 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 41498 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 41498 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 41498 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 42014 T118 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/collection1
[junit4:junit4] 2> 42014 T118 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 42015 T118 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 42016 T118 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 42018 T118 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/collection1/'
[junit4:junit4] 2> 42019 T118 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/collection1/lib/README' to classloader
[junit4:junit4] 2> 42020 T118 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 42070 T118 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 42144 T118 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 42146 T118 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 42185 T118 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 42861 T118 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 42876 T118 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 42880 T118 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 42912 T118 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 42918 T118 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 42924 T118 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 42925 T118 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 42926 T118 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 42926 T118 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 42928 T118 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 42928 T118 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 42929 T118 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 42929 T118 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371606061972/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/
[junit4:junit4] 2> 42929 T118 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 42930 T118 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 42931 T118 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4
[junit4:junit4] 2> 42932 T118 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index/
[junit4:junit4] 2> 42933 T118 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 42934 T118 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index
[junit4:junit4] 2> 42939 T118 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 42940 T118 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 42943 T118 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 42944 T118 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 42944 T118 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 42946 T118 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 42947 T118 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 42947 T118 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 42948 T118 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 42949 T118 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 42950 T118 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 42963 T118 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 42971 T118 oass.SolrIndexSearcher.<init> Opening Searcher@31ca9f31 main
[junit4:junit4] 2> 42972 T118 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 42972 T118 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 42982 T119 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31ca9f31 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 42987 T118 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 42987 T118 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63848 collection:collection1 shard:shard1
[junit4:junit4] 2> 42991 T118 oasc.ZkController.register We are http://127.0.0.1:63848/collection1/ and leader is http://127.0.0.1:39436/collection1/
[junit4:junit4] 2> 42991 T118 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63848
[junit4:junit4] 2> 42991 T118 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 42992 T118 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@41b21326 url=http://127.0.0.1:63848/collection1 node=127.0.0.1:63848_ C1_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63848_, base_url=http://127.0.0.1:63848}
[junit4:junit4] 2> 42998 T120 C1 P63848 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 42999 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 42999 T118 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 42999 T120 C1 P63848 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 43000 T120 C1 P63848 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 43001 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 43002 T120 C1 P63848 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 43002 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 43003 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 43004 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 43005 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63848__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63848"}
[junit4:junit4] 2> 43029 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43030 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43030 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43029 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43029 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43029 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 43046 T59 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:63848__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 43047 T59 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:63848__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4:junit4] 2> 43047 T59 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:63848__collection1&state=recovering&nodeName=127.0.0.1:63848_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=8
[junit4:junit4] 2> 43312 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 43316 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26355
[junit4:junit4] 2> 43317 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 43318 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 43318 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533
[junit4:junit4] 2> 43319 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/solr.xml
[junit4:junit4] 2> 43320 T27 oasc.CoreContainer.<init> New CoreContainer 2140337373
[junit4:junit4] 2> 43320 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/'
[junit4:junit4] 2> 43321 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/'
[junit4:junit4] 2> 43461 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 43462 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 43462 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 43463 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 43464 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 43464 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 43465 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 43466 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 43466 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 43467 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 43485 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 43486 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 43487 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 43488 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 43492 T132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f3ddd13 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 43492 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 43495 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 43509 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 43520 T134 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3498f5f9 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 43521 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 43527 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 44533 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26355_
[junit4:junit4] 2> 44558 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26355_
[junit4:junit4] 2> 44570 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 44570 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 44571 T134 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44570 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 44571 T101 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44571 T69 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44571 T85 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44573 T56 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44573 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 44573 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 44574 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 44601 T117 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44601 T49 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 44608 T135 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 44609 T135 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C1_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:63848_, base_url=http://127.0.0.1:63848}
[junit4:junit4] 2> 45049 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:39436/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 45061 T120 C1 P63848 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 45083 T120 C1 P63848 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:63848 START replicas=[http://127.0.0.1:39436/collection1/] nUpdates=100
[junit4:junit4] 2> 45096 T120 C1 P63848 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 45097 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 45098 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 45098 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 45099 T120 C1 P63848 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 45099 T120 C1 P63848 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:39436/collection1/. core=collection1
[junit4:junit4] 2> 45100 T120 C1 P63848 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@7a25a893 url=http://127.0.0.1:39436/collection1 node=127.0.0.1:39436_ C2_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39436_, base_url=http://127.0.0.1:39436, leader=true}
[junit4:junit4] 2> 45150 T61 C2 P39436 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 45199 T60 C2 P39436 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 45205 T60 C2 P39436 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 45206 T60 C2 P39436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 45210 T60 C2 P39436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 45211 T60 C2 P39436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 45214 T60 C2 P39436 oass.SolrIndexSearcher.<init> Opening Searcher@4fe2355f realtime
[junit4:junit4] 2> 45214 T60 C2 P39436 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 45215 T60 C2 P39436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 66
[junit4:junit4] 2> 45290 T120 C1 P63848 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 45291 T120 C1 P63848 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 45322 T120 C1 P63848 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 45326 T62 C2 P39436 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 45327 T62 C2 P39436 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 45329 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 45329 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 45330 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 45335 T62 C2 P39436 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 45337 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 45340 T120 C1 P63848 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846
[junit4:junit4] 2> 45341 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846 lockFactory=org.apache.lucene.store.NativeFSLockFactory@317fd5ad fullCopy=false
[junit4:junit4] 2> 45358 T62 C2 P39436 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6
[junit4:junit4] 2> 45376 T120 C1 P63848 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 45379 T120 C1 P63848 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 45380 T120 C1 P63848 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 45392 T120 C1 P63848 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 45393 T120 C1 P63848 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 45394 T120 C1 P63848 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 45395 T120 C1 P63848 oass.SolrIndexSearcher.<init> Opening Searcher@11a19fe5 main
[junit4:junit4] 2> 45408 T119 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11a19fe5 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 45410 T120 C1 P63848 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846;done=true>>]
[junit4:junit4] 2> 45411 T120 C1 P63848 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846
[junit4:junit4] 2> 45411 T120 C1 P63848 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty4/index.20130619064108846
[junit4:junit4] 2> 45421 T120 C1 P63848 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 45421 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 45422 T120 C1 P63848 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 45422 T120 C1 P63848 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 45424 T120 C1 P63848 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 46064 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 46066 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26355_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26355"}
[junit4:junit4] 2> 46066 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 46067 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 46074 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63848__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63848"}
[junit4:junit4] 2> 46090 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46090 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 46614 T135 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/collection1
[junit4:junit4] 2> 46614 T135 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 46616 T135 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 46616 T135 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 46619 T135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/collection1/'
[junit4:junit4] 2> 46621 T135 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 46622 T135 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/collection1/lib/README' to classloader
[junit4:junit4] 2> 46762 T135 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 46848 T135 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 46851 T135 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 46890 T135 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 47499 T135 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 47517 T135 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 47520 T135 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 47553 T135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 47560 T135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 47565 T135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 47567 T135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 47568 T135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 47568 T135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 47569 T135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 47570 T135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 47570 T135 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 47570 T135 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371606066533/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/
[junit4:junit4] 2> 47571 T135 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 47572 T135 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 47573 T135 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5
[junit4:junit4] 2> 47573 T135 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index/
[junit4:junit4] 2> 47574 T135 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 47575 T135 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index
[junit4:junit4] 2> 47580 T135 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 47580 T135 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 47584 T135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 47584 T135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 47585 T135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 47586 T135 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 47587 T135 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 47588 T135 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 47598 T135 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 47599 T135 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 47600 T135 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 47611 T135 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 47619 T135 oass.SolrIndexSearcher.<init> Opening Searcher@50ec4d43 main
[junit4:junit4] 2> 47621 T135 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 47621 T135 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 47630 T138 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50ec4d43 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 47635 T135 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 47635 T135 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26355 collection:collection1 shard:shard2
[junit4:junit4] 2> 47639 T135 oasc.ZkController.register We are http://127.0.0.1:26355/collection1/ and leader is http://127.0.0.1:45842/collection1/
[junit4:junit4] 2> 47639 T135 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26355
[junit4:junit4] 2> 47639 T135 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 47640 T135 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@5bfe1db2 url=http://127.0.0.1:26355/collection1 node=127.0.0.1:26355_ C3_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26355_, base_url=http://127.0.0.1:26355}
[junit4:junit4] 2> 47640 T139 C3 P26355 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 47641 T135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 47641 T139 C3 P26355 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 47642 T139 C3 P26355 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 47642 T139 C3 P26355 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 47643 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 47644 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 47644 T139 C3 P26355 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 47645 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 47666 T75 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:26355__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 47958 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 47961 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65091
[junit4:junit4] 2> 47962 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 47963 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 47964 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172
[junit4:junit4] 2> 47965 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/solr.xml
[junit4:junit4] 2> 47965 T27 oasc.CoreContainer.<init> New CoreContainer 2108957012
[junit4:junit4] 2> 47966 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/'
[junit4:junit4] 2> 47967 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/'
[junit4:junit4] 2> 48147 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 48148 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 48149 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 48150 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 48150 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 48151 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 48152 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 48152 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 48153 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 48154 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 48173 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 48174 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 48175 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 48176 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 48179 T151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@416ed704 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 48180 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 48183 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 48198 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 48200 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52cc6a6e name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 48201 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 48213 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 49107 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 49109 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:26355__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26355_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26355"}
[junit4:junit4] 2> 49123 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49123 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49218 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65091_
[junit4:junit4] 2> 49221 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65091_
[junit4:junit4] 2> 49225 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49225 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49226 T101 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49226 T69 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49225 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49225 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 49228 T56 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49227 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 49227 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 49226 T153 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49225 T85 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49230 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 49230 T117 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49230 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 49231 T134 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49232 T49 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 49238 T154 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 49239 T154 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 49669 T75 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:26355__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 49670 T75 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:26355__collection1&state=recovering&nodeName=127.0.0.1:26355_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 50631 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 50632 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:65091_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:65091"}
[junit4:junit4] 2> 50633 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 50633 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 50649 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 50650 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 51243 T154 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/collection1
[junit4:junit4] 2> 51243 T154 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 51245 T154 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 51245 T154 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 51248 T154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/collection1/'
[junit4:junit4] 2> 51250 T154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 51251 T154 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/collection1/lib/README' to classloader
[junit4:junit4] 2> 51317 T154 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 51391 T154 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 51394 T154 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 51423 T154 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@5bfe1db2 url=http://127.0.0.1:26355/collection1 node=127.0.0.1:26355_ C4_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26355_, base_url=http://127.0.0.1:26355}
[junit4:junit4] 2> 51672 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:45842/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 51672 T139 C4 P26355 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26355 START replicas=[http://127.0.0.1:45842/collection1/] nUpdates=100
[junit4:junit4] 2> 51673 T139 C4 P26355 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 51674 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 51674 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 51675 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 51675 T139 C4 P26355 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 51676 T139 C4 P26355 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:45842/collection1/. core=collection1
[junit4:junit4] 2> 51677 T139 C4 P26355 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@19bfcd9f url=http://127.0.0.1:45842/collection1 node=127.0.0.1:45842_ C5_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:45842_, base_url=http://127.0.0.1:45842, leader=true}
[junit4:junit4] 2> 51685 T76 C5 P45842 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 51695 T77 C5 P45842 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 51702 T77 C5 P45842 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 51702 T77 C5 P45842 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 51708 T77 C5 P45842 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 51709 T77 C5 P45842 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 51711 T77 C5 P45842 oass.SolrIndexSearcher.<init> Opening Searcher@66ff56b6 realtime
[junit4:junit4] 2> 51712 T77 C5 P45842 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 51712 T77 C5 P45842 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 51714 T139 C4 P26355 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 51715 T139 C4 P26355 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 51718 T78 C5 P45842 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 51719 T78 C5 P45842 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 51721 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 51721 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 51722 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 51726 T78 C5 P45842 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 51727 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 51729 T139 C4 P26355 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236
[junit4:junit4] 2> 51731 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe184a1 fullCopy=false
[junit4:junit4] 2> 51736 T78 C5 P45842 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 51739 T139 C4 P26355 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 51743 T139 C4 P26355 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 51743 T139 C4 P26355 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 51750 T139 C4 P26355 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 51750 T139 C4 P26355 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 51751 T139 C4 P26355 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 51753 T139 C4 P26355 oass.SolrIndexSearcher.<init> Opening Searcher@5c924417 main
[junit4:junit4] 2> 51756 T138 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c924417 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 51757 T139 C4 P26355 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236;done=true>>]
[junit4:junit4] 2> 51758 T139 C4 P26355 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236
[junit4:junit4] 2> 51758 T139 C4 P26355 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty5/index.20130619064115236
[junit4:junit4] 2> 51760 T139 C4 P26355 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 51760 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 51761 T139 C4 P26355 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 51761 T139 C4 P26355 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 51763 T139 C4 P26355 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 52158 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 52160 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:26355__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26355_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26355"}
[junit4:junit4] 2> 52176 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52176 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52177 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 52204 T154 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 52224 T154 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 52229 T154 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 52267 T154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 52292 T154 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 52299 T154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 52301 T154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 52301 T154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 52302 T154 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 52304 T154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 52304 T154 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 52305 T154 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 52306 T154 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371606071172/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6/
[junit4:junit4] 2> 52306 T154 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@114379f5
[junit4:junit4] 2> 52307 T154 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 52309 T154 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6
[junit4:junit4] 2> 52310 T154 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6/index/
[junit4:junit4] 2> 52311 T154 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 52312 T154 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6/index
[junit4:junit4] 2> 52319 T154 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371606033768/jetty6/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 52320 T154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 52325 T154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 52325 T154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 52326 T154 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 52328 T154 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 52329 T154 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 52330 T154 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 52334 T154 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 52336 T154 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 52336 T154 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 52356 T154 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 52367 T154 oass.SolrIndexSearcher.<init> Opening Searcher@2bffef5a main
[junit4:junit4] 2> 52369 T154 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 52369 T154 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 52381 T157 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bffef5a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 52388 T154 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 52388 T154 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65091 collection:collection1 shard:shard3
[junit4:junit4] 2> 52394 T154 oasc.ZkController.register We are http://127.0.0.1:65091/collection1/ and leader is http://127.0.0.1:53771/collection1/
[junit4:junit4] 2> 52394 T154 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65091
[junit4:junit4] 2> 52395 T154 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 52395 T154 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@41282ad4 url=http://127.0.0.1:65091/collection1 node=127.0.0.1:65091_ C6_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:65091_, base_url=http://127.0.0.1:65091}
[junit4:junit4] 2> 52396 T158 C6 P65091 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 52397 T158 C6 P65091 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 52397 T154 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 52397 T158 C6 P65091 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 52398 T158 C6 P65091 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 52399 T27 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 52400 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 52400 T158 C6 P65091 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 52401 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 52426 T91 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:65091__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 52710 T27 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 52714 T27 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52388
[junit4:junit4] 2> 52715 T27 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 52715 T27 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 52716 T27 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371606075931
[junit4:junit4] 2> 52717 T27 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371606075931/solr.xml
[junit4:junit4] 2> 52717 T27 oasc.CoreContainer.<init> New CoreContainer 1631091236
[junit4:junit4] 2> 52718 T27 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371606075931/'
[junit4:junit4] 2> 52719 T27 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371606075931/'
[junit4:junit4] 2> 52873 T27 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 52874 T27 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 52875 T27 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 52875 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 52876 T27 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 52877 T27 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 52877 T27 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 52878 T27 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 52878 T27 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 52879 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 52902 T27 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 52902 T27 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60574/solr
[junit4:junit4] 2> 52903 T27 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 52904 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 52910 T170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a411470 name:ZooKeeperConnection Watcher:127.0.0.1:60574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 52911 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 52925 T27 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 52940 T27 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 52943 T172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a183786 name:ZooKeeperConnection Watcher:127.0.0.1:60574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 52943 T27 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 52956 T27 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 53685 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 53687 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:65091__collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:65091_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:65091"}
[junit4:junit4] 2> 53706 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53706 T172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53962 T27 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52388_
[junit4:junit4] 2> 53964 T27 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52388_
[junit4:junit4] 2> 53968 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53968 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53968 T134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53968 T172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53968 T49 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 53969 T101 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53969 T69 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53969 T153 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53969 T85 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53972 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 53972 T117 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53972 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 53972 T56 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53971 T101 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 53974 T49 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53974 T172 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53974 T134 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 53973 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 53982 T173 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 53983 T173 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 54429 T91 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:65091__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 54430 T91 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:65091__collection1&state=recovering&nodeName=127.0.0.1:65091_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 55214 T50 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 55215 T50 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:52388_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52388"}
[junit4:junit4] 2> 55216 T50 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 55216 T50 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 55232 T56 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 55232 T69 oascc.ZkSt
[...truncated too long message...]
2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 921545 T1081 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 921545 T1081 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 921545 T1081 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 921546 T1081 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 921546 T1081 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 921546 T1081 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 922609 T1098 oasu.PeerSync.handleResponse WARN PeerSync: core=collection1 url=http://127.0.0.1:64221 couldn't connect to http://127.0.0.1:64207/collection1/, counting as success
[junit4:junit4] 2> 922609 T1098 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:64221 DONE. sync succeeded
[junit4:junit4] 2> 922609 T1098 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 922610 T1098 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4:junit4] 2> 922610 T1098 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@68997159
[junit4:junit4] 2> 922618 T1098 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=590,adds=590,deletesById=295,deletesByQuery=0,errors=0,cumulative_adds=632,cumulative_deletesById=295,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 922619 T1098 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 922619 T1098 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 922620 T1098 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> ASYNC NEW_CORE C779 name=collection1 org.apache.solr.core.SolrCore@68997159 url=http://127.0.0.1:64221/collection1 node=127.0.0.1:64221_ C779_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:64221_, base_url=http://127.0.0.1:64221}
[junit4:junit4] 2> 922641 T1098 C779 P64221 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2fffbd21 lockFactory=org.apache.lucene.store.NativeFSLockFactory@39b52b43),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2fffbd21 lockFactory=org.apache.lucene.store.NativeFSLockFactory@39b52b43),segFN=segments_5,generation=5}
[junit4:junit4] 2> 922642 T1098 C779 P64221 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 922646 T1098 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 922647 T1098 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 922647 T1098 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4/index;done=false>>]
[junit4:junit4] 2> 922647 T1098 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4/index
[junit4:junit4] 2> 922647 T1098 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4;done=false>>]
[junit4:junit4] 2> 922648 T1098 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371606724704/jetty4
[junit4:junit4] 2> 922648 T1098 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 922648 T1098 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 922649 T1098 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 922649 T1098 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=pl_PL, timezone=Etc/GMT-8
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=70191288,total=164626432
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestSolrCoreProperties, ChaosMonkeySafeLeaderTest, RegexBoostProcessorTest, DebugComponentTest, StandardRequestHandlerTest, DateMathParserTest, SampleTest, UpdateParamsTest, JSONWriterTest, TestUniqueKeyFieldResource, IndexSchemaRuntimeFieldTest, ReturnFieldsTest, SpellPossibilityIteratorTest, MinimalSchemaTest, ClusterStateTest, DistributedTermsComponentTest, TestSolrIndexConfig, AutoCommitTest, SpatialFilterTest, DateFieldTest, SolrRequestParserTest, TimeZoneUtilsTest, TestDefaultSimilarityFactory, PathHierarchyTokenizerFactoryTest, TestQuerySenderListener, ShardSplitTest]
[junit4:junit4] Completed on J1 in 221.76s, 1 test, 1 error <<< FAILURES!
[...truncated 626 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1246: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:890: There were test failures: 298 suites, 1253 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 54 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1345 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1345/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:50758 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:50758 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([32E903CEE0C7A1D2:B30F8DD69798C1EE]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9695 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 1055749 T2208 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1055757 T2208 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1371726051130
[junit4:junit4] 2> 1055759 T2208 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1055760 T2209 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1055860 T2208 oasc.ZkTestServer.run start zk server on port:49106
[junit4:junit4] 2> 1055861 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1055871 T2215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d9594b name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1055871 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1055872 T2208 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1055888 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1055890 T2217 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34fa3e3f name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1055890 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1055891 T2208 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1055900 T2208 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1055902 T2208 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1055911 T2208 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1055914 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1055914 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1055926 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1055927 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1056030 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1056031 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1056044 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1056045 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1056048 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1056049 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1056052 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1056053 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1056056 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1056057 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1056059 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1056060 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1056063 T2208 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1056064 T2208 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1056342 T2208 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1056348 T2208 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44754
[junit4:junit4] 2> 1056349 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1056350 T2208 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1056350 T2208 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446
[junit4:junit4] 2> 1056351 T2208 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/solr.xml
[junit4:junit4] 2> 1056351 T2208 oasc.CoreContainer.<init> New CoreContainer 1290646299
[junit4:junit4] 2> 1056352 T2208 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/'
[junit4:junit4] 2> 1056353 T2208 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/'
[junit4:junit4] 2> 1056485 T2208 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1056486 T2208 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1056486 T2208 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1056487 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1056487 T2208 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1056488 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1056489 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1056489 T2208 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1056490 T2208 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1056490 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1056508 T2208 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1056509 T2208 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49106/solr
[junit4:junit4] 2> 1056510 T2208 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1056511 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1056514 T2228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fffe529 name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1056514 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1056517 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1056529 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1056531 T2230 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ceed746 name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1056531 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1056533 T2208 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1056542 T2208 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1056546 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1056553 T2208 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44754_
[junit4:junit4] 2> 1056555 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44754_
[junit4:junit4] 2> 1056558 T2208 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1056574 T2208 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1056577 T2208 oasc.Overseer.start Overseer (id=89897438487379971-127.0.0.1:44754_-n_0000000000) starting
[junit4:junit4] 2> 1056587 T2208 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1056598 T2232 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1056598 T2208 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1056601 T2208 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1056608 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1056610 T2231 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1056615 T2233 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1056615 T2233 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1058113 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1058114 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44754_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44754"}
[junit4:junit4] 2> 1058115 T2231 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1058115 T2231 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1058164 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1058618 T2233 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/collection1
[junit4:junit4] 2> 1058618 T2233 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1058619 T2233 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1058619 T2233 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1058621 T2233 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/collection1/'
[junit4:junit4] 2> 1058622 T2233 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1058623 T2233 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/collection1/lib/README' to classloader
[junit4:junit4] 2> 1058677 T2233 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1058739 T2233 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1058840 T2233 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1058845 T2233 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1059437 T2233 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1059441 T2233 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1059443 T2233 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1059449 T2233 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1059472 T2233 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1059472 T2233 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371726051446/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data/
[junit4:junit4] 2> 1059473 T2233 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62bc454f
[junit4:junit4] 2> 1059473 T2233 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1059474 T2233 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data
[junit4:junit4] 2> 1059475 T2233 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data/index/
[junit4:junit4] 2> 1059475 T2233 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1059475 T2233 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data/index
[junit4:junit4] 2> 1059480 T2233 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cc34d62 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4054302b)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1059481 T2233 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1059484 T2233 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1059484 T2233 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1059485 T2233 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1059486 T2233 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1059486 T2233 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1059487 T2233 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1059487 T2233 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1059488 T2233 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1059488 T2233 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1059505 T2233 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1059512 T2233 oass.SolrIndexSearcher.<init> Opening Searcher@1f6e8422 main
[junit4:junit4] 2> 1059513 T2233 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/control/data/tlog
[junit4:junit4] 2> 1059514 T2233 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1059514 T2233 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1059518 T2234 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f6e8422 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1059520 T2233 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1059520 T2233 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44754 collection:control_collection shard:shard1
[junit4:junit4] 2> 1059521 T2233 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1059547 T2233 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1059558 T2233 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1059558 T2233 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1059559 T2233 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44754/collection1/
[junit4:junit4] 2> 1059559 T2233 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1059559 T2233 oasc.SyncStrategy.syncToMe http://127.0.0.1:44754/collection1/ has no replicas
[junit4:junit4] 2> 1059560 T2233 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44754/collection1/
[junit4:junit4] 2> 1059560 T2233 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1059668 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1059687 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1059725 T2233 oasc.ZkController.register We are http://127.0.0.1:44754/collection1/ and leader is http://127.0.0.1:44754/collection1/
[junit4:junit4] 2> 1059725 T2233 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44754
[junit4:junit4] 2> 1059725 T2233 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1059726 T2233 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1059726 T2233 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1059728 T2233 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1059729 T2208 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1059729 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1059730 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1059748 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1059749 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1059751 T2237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@243306de name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1059752 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1059753 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1059755 T2208 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1060030 T2208 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1060034 T2208 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50758
[junit4:junit4] 2> 1060035 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1060035 T2208 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1060036 T2208 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128
[junit4:junit4] 2> 1060036 T2208 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/solr.xml
[junit4:junit4] 2> 1060037 T2208 oasc.CoreContainer.<init> New CoreContainer 897652382
[junit4:junit4] 2> 1060038 T2208 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/'
[junit4:junit4] 2> 1060039 T2208 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/'
[junit4:junit4] 2> 1060170 T2208 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1060171 T2208 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1060172 T2208 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1060172 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1060173 T2208 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1060173 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1060174 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1060174 T2208 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1060175 T2208 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1060176 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1060193 T2208 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1060194 T2208 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49106/solr
[junit4:junit4] 2> 1060195 T2208 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1060196 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1060199 T2248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6565806d name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1060199 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1060214 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1060226 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1060228 T2250 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c5ba35 name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1060228 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1060238 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1061192 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1061193 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44754__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44754_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44754"}
[junit4:junit4] 2> 1061208 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1061208 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1061208 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1061241 T2208 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50758_
[junit4:junit4] 2> 1061243 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50758_
[junit4:junit4] 2> 1061245 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1061245 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1061246 T2250 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1061246 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1061247 T2230 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1061255 T2237 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1061261 T2251 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1061261 T2251 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1062713 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1062714 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50758_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50758"}
[junit4:junit4] 2> 1062714 T2231 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1062714 T2231 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1062729 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1062729 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1062729 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1063264 T2251 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/collection1
[junit4:junit4] 2> 1063264 T2251 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1063265 T2251 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1063265 T2251 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1063267 T2251 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/collection1/'
[junit4:junit4] 2> 1063269 T2251 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/collection1/lib/README' to classloader
[junit4:junit4] 2> 1063269 T2251 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1063323 T2251 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1063385 T2251 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1063486 T2251 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1063492 T2251 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1064080 T2251 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1064084 T2251 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1064086 T2251 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1064092 T2251 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1064115 T2251 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1064115 T2251 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371726055128/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1/
[junit4:junit4] 2> 1064116 T2251 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62bc454f
[junit4:junit4] 2> 1064116 T2251 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1064117 T2251 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1
[junit4:junit4] 2> 1064118 T2251 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1/index/
[junit4:junit4] 2> 1064118 T2251 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1064120 T2251 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1/index
[junit4:junit4] 2> 1064124 T2251 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1064124 T2251 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1064128 T2251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1064128 T2251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1064129 T2251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1064130 T2251 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1064131 T2251 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1064131 T2251 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1064131 T2251 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1064132 T2251 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1064133 T2251 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1064148 T2251 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1064156 T2251 oass.SolrIndexSearcher.<init> Opening Searcher@754aa956 main
[junit4:junit4] 2> 1064156 T2251 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty1/tlog
[junit4:junit4] 2> 1064157 T2251 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1064157 T2251 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1064162 T2252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@754aa956 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1064164 T2251 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1064165 T2251 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50758 collection:collection1 shard:shard1
[junit4:junit4] 2> 1064166 T2251 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1064191 T2251 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1064202 T2251 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1064202 T2251 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1064203 T2251 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50758/collection1/
[junit4:junit4] 2> 1064203 T2251 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1064203 T2251 oasc.SyncStrategy.syncToMe http://127.0.0.1:50758/collection1/ has no replicas
[junit4:junit4] 2> 1064204 T2251 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50758/collection1/
[junit4:junit4] 2> 1064204 T2251 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1064233 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1064251 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1064251 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1064251 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1064261 T2251 oasc.ZkController.register We are http://127.0.0.1:50758/collection1/ and leader is http://127.0.0.1:50758/collection1/
[junit4:junit4] 2> 1064261 T2251 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50758
[junit4:junit4] 2> 1064261 T2251 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1064261 T2251 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1064262 T2251 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1064263 T2251 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1064264 T2208 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1064265 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1064266 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1064527 T2208 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1064531 T2208 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57978
[junit4:junit4] 2> 1064531 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1064532 T2208 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1064532 T2208 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654
[junit4:junit4] 2> 1064533 T2208 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/solr.xml
[junit4:junit4] 2> 1064534 T2208 oasc.CoreContainer.<init> New CoreContainer 1401328295
[junit4:junit4] 2> 1064534 T2208 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/'
[junit4:junit4] 2> 1064535 T2208 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/'
[junit4:junit4] 2> 1064655 T2208 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1064656 T2208 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1064657 T2208 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1064657 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1064658 T2208 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1064658 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1064659 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1064659 T2208 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1064660 T2208 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1064660 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1064677 T2208 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1064677 T2208 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49106/solr
[junit4:junit4] 2> 1064678 T2208 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1064679 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1064682 T2264 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a0aa8d name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1064683 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1064701 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1064713 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1064715 T2266 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2737b902 name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1064715 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1064725 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1065728 T2208 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57978_
[junit4:junit4] 2> 1065730 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57978_
[junit4:junit4] 2> 1065733 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1065733 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1065733 T2250 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1065733 T2266 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1065734 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1065745 T2237 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1065745 T2230 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1065751 T2267 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1065752 T2267 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1065756 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1065757 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50758__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50758_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50758"}
[junit4:junit4] 2> 1065761 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57978_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57978"}
[junit4:junit4] 2> 1065761 T2231 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1065761 T2231 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1065775 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1065775 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1065775 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1065775 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1066754 T2267 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/collection1
[junit4:junit4] 2> 1066754 T2267 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1066755 T2267 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1066755 T2267 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1066757 T2267 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/collection1/'
[junit4:junit4] 2> 1066758 T2267 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/collection1/lib/README' to classloader
[junit4:junit4] 2> 1066759 T2267 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1066813 T2267 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1066874 T2267 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1066976 T2267 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1066982 T2267 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1067566 T2267 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1067571 T2267 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1067573 T2267 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1067578 T2267 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1067601 T2267 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1067601 T2267 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371726059654/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2/
[junit4:junit4] 2> 1067601 T2267 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62bc454f
[junit4:junit4] 2> 1067602 T2267 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1067604 T2267 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2
[junit4:junit4] 2> 1067604 T2267 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2/index/
[junit4:junit4] 2> 1067604 T2267 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1067605 T2267 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2/index
[junit4:junit4] 2> 1067609 T2267 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1067609 T2267 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1067613 T2267 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1067613 T2267 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1067614 T2267 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1067615 T2267 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1067616 T2267 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1067616 T2267 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1067617 T2267 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1067617 T2267 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1067618 T2267 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1067634 T2267 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1067641 T2267 oass.SolrIndexSearcher.<init> Opening Searcher@4a52ef4 main
[junit4:junit4] 2> 1067641 T2267 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty2/tlog
[junit4:junit4] 2> 1067643 T2267 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1067643 T2267 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1067648 T2268 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a52ef4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1067650 T2267 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1067650 T2267 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57978 collection:collection1 shard:shard2
[junit4:junit4] 2> 1067651 T2267 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1067668 T2267 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1067671 T2267 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1067671 T2267 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1067671 T2267 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57978/collection1/
[junit4:junit4] 2> 1067672 T2267 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1067672 T2267 oasc.SyncStrategy.syncToMe http://127.0.0.1:57978/collection1/ has no replicas
[junit4:junit4] 2> 1067672 T2267 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57978/collection1/
[junit4:junit4] 2> 1067672 T2267 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1068783 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1068819 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1068819 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1068819 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1068819 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1068850 T2267 oasc.ZkController.register We are http://127.0.0.1:57978/collection1/ and leader is http://127.0.0.1:57978/collection1/
[junit4:junit4] 2> 1068850 T2267 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57978
[junit4:junit4] 2> 1068850 T2267 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1068851 T2267 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1068851 T2267 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1068852 T2267 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1068854 T2208 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1068854 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1068855 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1069139 T2208 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1069143 T2208 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35209
[junit4:junit4] 2> 1069144 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1069144 T2208 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1069145 T2208 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243
[junit4:junit4] 2> 1069146 T2208 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/solr.xml
[junit4:junit4] 2> 1069146 T2208 oasc.CoreContainer.<init> New CoreContainer 166053106
[junit4:junit4] 2> 1069147 T2208 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/'
[junit4:junit4] 2> 1069148 T2208 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/'
[junit4:junit4] 2> 1069277 T2208 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1069278 T2208 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1069279 T2208 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1069279 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1069280 T2208 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1069280 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1069281 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1069281 T2208 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1069282 T2208 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1069283 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1069298 T2208 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1069299 T2208 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49106/solr
[junit4:junit4] 2> 1069299 T2208 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1069301 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1069304 T2280 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7151c1a7 name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1069305 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1069318 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1069329 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1069331 T2282 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f4894ea name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1069331 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1069336 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1070334 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1070335 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57978__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57978_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57978"}
[junit4:junit4] 2> 1070339 T2208 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35209_
[junit4:junit4] 2> 1070353 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070353 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070353 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070354 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35209_
[junit4:junit4] 2> 1070353 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070353 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070362 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070362 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070362 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1070362 T2282 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1070362 T2250 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1070363 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1070363 T2266 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1070363 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1070369 T2230 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1070369 T2237 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1070375 T2283 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1070375 T2283 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1071862 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1071863 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35209_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35209"}
[junit4:junit4] 2> 1071864 T2231 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1071864 T2231 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1071879 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1071879 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1071879 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1071879 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1071879 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1072379 T2283 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/collection1
[junit4:junit4] 2> 1072379 T2283 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1072380 T2283 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1072380 T2283 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1072382 T2283 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/collection1/'
[junit4:junit4] 2> 1072383 T2283 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1072384 T2283 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/collection1/lib/README' to classloader
[junit4:junit4] 2> 1072439 T2283 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1072500 T2283 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1072601 T2283 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1072607 T2283 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1073192 T2283 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1073196 T2283 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1073198 T2283 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1073203 T2283 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1073225 T2283 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1073225 T2283 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371726064243/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/
[junit4:junit4] 2> 1073225 T2283 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62bc454f
[junit4:junit4] 2> 1073226 T2283 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1073227 T2283 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3
[junit4:junit4] 2> 1073227 T2283 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index/
[junit4:junit4] 2> 1073227 T2283 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1073229 T2283 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index
[junit4:junit4] 2> 1073232 T2283 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@455d4e2f lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a15d8b)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1073233 T2283 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1073236 T2283 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1073236 T2283 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1073237 T2283 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1073238 T2283 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1073239 T2283 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1073239 T2283 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1073239 T2283 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1073240 T2283 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1073241 T2283 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1073256 T2283 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1073262 T2283 oass.SolrIndexSearcher.<init> Opening Searcher@389dfe8e main
[junit4:junit4] 2> 1073263 T2283 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/tlog
[junit4:junit4] 2> 1073267 T2283 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1073267 T2283 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1073280 T2284 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@389dfe8e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1073282 T2283 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1073282 T2283 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35209 collection:collection1 shard:shard1
[junit4:junit4] 2> 1073288 T2283 oasc.ZkController.register We are http://127.0.0.1:35209/collection1/ and leader is http://127.0.0.1:50758/collection1/
[junit4:junit4] 2> 1073288 T2283 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35209
[junit4:junit4] 2> 1073288 T2283 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1073289 T2283 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C344 name=collection1 org.apache.solr.core.SolrCore@fd4aa85 url=http://127.0.0.1:35209/collection1 node=127.0.0.1:35209_ C344_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:35209_, base_url=http://127.0.0.1:35209}
[junit4:junit4] 2> 1073289 T2285 C344 P35209 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1073289 T2283 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1073290 T2285 C344 P35209 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1073290 T2285 C344 P35209 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1073291 T2285 C344 P35209 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1073291 T2208 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1073292 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1073292 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1073298 T2285 C344 P35209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1073316 T2240 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:35209__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1073383 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1073384 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35209__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35209_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35209"}
[junit4:junit4] 2> 1073402 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1073403 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1073403 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1073403 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1073402 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1073588 T2208 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1073592 T2208 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25724
[junit4:junit4] 2> 1073593 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1073594 T2208 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1073594 T2208 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686
[junit4:junit4] 2> 1073595 T2208 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/solr.xml
[junit4:junit4] 2> 1073595 T2208 oasc.CoreContainer.<init> New CoreContainer 1074282113
[junit4:junit4] 2> 1073596 T2208 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/'
[junit4:junit4] 2> 1073597 T2208 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/'
[junit4:junit4] 2> 1073727 T2208 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1073728 T2208 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1073728 T2208 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1073729 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1073729 T2208 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1073730 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1073730 T2208 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1073731 T2208 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1073732 T2208 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1073732 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1073750 T2208 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1073750 T2208 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49106/solr
[junit4:junit4] 2> 1073751 T2208 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1073752 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1073755 T2297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48c534b8 name:ZooKeeperConnection Watcher:127.0.0.1:49106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1073756 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1073766 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1073777 T2208 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1073779 T2299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@281ce414 name:ZooKeeperConnection Watcher:127.0.0.1:49106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1073779 T2208 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1073791 T2208 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1074317 T2240 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:35209__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1074317 T2240 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:35209__collection1&state=recovering&nodeName=127.0.0.1:35209_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 1074794 T2208 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25724_
[junit4:junit4] 2> 1074796 T2208 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25724_
[junit4:junit4] 2> 1074799 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1074799 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1074799 T2299 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074799 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1074799 T2282 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074799 T2250 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074800 T2237 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074800 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074800 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074813 T2230 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074813 T2266 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1074819 T2300 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1074819 T2300 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1074907 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1074908 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25724_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25724"}
[junit4:junit4] 2> 1074908 T2231 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1074908 T2231 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1074920 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074920 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074920 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074920 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074920 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1074920 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1075822 T2300 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/collection1
[junit4:junit4] 2> 1075822 T2300 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1075823 T2300 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1075823 T2300 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1075825 T2300 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/collection1/'
[junit4:junit4] 2> 1075826 T2300 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/collection1/lib/README' to classloader
[junit4:junit4] 2> 1075827 T2300 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1075881 T2300 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1075943 T2300 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1076044 T2300 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1076050 T2300 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C345 name=collection1 org.apache.solr.core.SolrCore@fd4aa85 url=http://127.0.0.1:35209/collection1 node=127.0.0.1:35209_ C345_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:35209_, base_url=http://127.0.0.1:35209}
[junit4:junit4] 2> 1076319 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50758/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1076320 T2285 C345 P35209 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35209 START replicas=[http://127.0.0.1:50758/collection1/] nUpdates=100
[junit4:junit4] 2> 1076320 T2285 C345 P35209 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1076321 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1076321 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1076321 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1076322 T2285 C345 P35209 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1076322 T2285 C345 P35209 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50758/collection1/. core=collection1
[junit4:junit4] 2> 1076322 T2285 C345 P35209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C346 name=collection1 org.apache.solr.core.SolrCore@31d9a4b3 url=http://127.0.0.1:50758/collection1 node=127.0.0.1:50758_ C346_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50758_, base_url=http://127.0.0.1:50758, leader=true}
[junit4:junit4] 2> 1076329 T2241 C346 P50758 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1076336 T2242 C346 P50758 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1076340 T2242 C346 P50758 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1076340 T2242 C346 P50758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1076341 T2242 C346 P50758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1076342 T2242 C346 P50758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1076342 T2242 C346 P50758 oass.SolrIndexSearcher.<init> Opening Searcher@3a222f1b realtime
[junit4:junit4] 2> 1076343 T2242 C346 P50758 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1076343 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1076344 T2285 C345 P35209 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1076345 T2285 C345 P35209 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1076347 T2243 C346 P50758 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1076348 T2243 C346 P50758 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1076349 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1076349 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1076349 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1076352 T2243 C346 P50758 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1076353 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1076354 T2285 C345 P35209 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index.20130620153111725
[junit4:junit4] 2> 1076354 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b1ec1b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69418ca4) fullCopy=false
[junit4:junit4] 2> 1076357 T2243 C346 P50758 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1076358 T2285 C345 P35209 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1076360 T2285 C345 P35209 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1076360 T2285 C345 P35209 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1076362 T2285 C345 P35209 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@455d4e2f lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a15d8b)),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@455d4e2f lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a15d8b)),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1076363 T2285 C345 P35209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1076363 T2285 C345 P35209 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1076364 T2285 C345 P35209 oass.SolrIndexSearcher.<init> Opening Searcher@6993c5c8 main
[junit4:junit4] 2> 1076365 T2284 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6993c5c8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1076366 T2285 C345 P35209 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index.20130620153111725 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index.20130620153111725;done=true>>]
[junit4:junit4] 2> 1076366 T2285 C345 P35209 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index.20130620153111725
[junit4:junit4] 2> 1076366 T2285 C345 P35209 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty3/index.20130620153111725
[junit4:junit4] 2> 1076367 T2285 C345 P35209 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1076367 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1076367 T2285 C345 P35209 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1076367 T2285 C345 P35209 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1076369 T2285 C345 P35209 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1076424 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1076425 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35209__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35209_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35209"}
[junit4:junit4] 2> 1076438 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076438 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076438 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076438 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076438 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076438 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1076638 T2300 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1076641 T2300 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1076642 T2300 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1076646 T2300 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1076666 T2300 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1076666 T2300 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371726068686/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/
[junit4:junit4] 2> 1076666 T2300 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62bc454f
[junit4:junit4] 2> 1076667 T2300 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1076667 T2300 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4
[junit4:junit4] 2> 1076668 T2300 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index/
[junit4:junit4] 2> 1076668 T2300 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1076669 T2300 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index
[junit4:junit4] 2> 1076671 T2300 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@778c59bf lockFactory=org.apache.lucene.store.NativeFSLockFactory@4422364e)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1076672 T2300 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1076674 T2300 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1076674 T2300 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1076675 T2300 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1076676 T2300 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1076676 T2300 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1076676 T2300 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1076677 T2300 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1076678 T2300 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1076678 T2300 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1076689 T2300 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1076694 T2300 oass.SolrIndexSearcher.<init> Opening Searcher@2c9bdd53 main
[junit4:junit4] 2> 1076695 T2300 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/tlog
[junit4:junit4] 2> 1076696 T2300 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1076696 T2300 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1076700 T2303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c9bdd53 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1076702 T2300 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1076702 T2300 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25724 collection:collection1 shard:shard2
[junit4:junit4] 2> 1076705 T2300 oasc.ZkController.register We are http://127.0.0.1:25724/collection1/ and leader is http://127.0.0.1:57978/collection1/
[junit4:junit4] 2> 1076705 T2300 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25724
[junit4:junit4] 2> 1076705 T2300 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1076705 T2300 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C347 name=collection1 org.apache.solr.core.SolrCore@41ffadad url=http://127.0.0.1:25724/collection1 node=127.0.0.1:25724_ C347_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:25724_, base_url=http://127.0.0.1:25724}
[junit4:junit4] 2> 1076706 T2304 C347 P25724 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1076706 T2300 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1076706 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1076707 T2304 C347 P25724 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1076707 T2304 C347 P25724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1076708 T2208 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1076708 T2208 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1076709 T2304 C347 P25724 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1076709 T2208 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1076730 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1076731 T2256 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:25724__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1076731 T2208 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1076732 T2208 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1076732 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1077734 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1077942 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1077943 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:25724__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25724_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25724"}
[junit4:junit4] 2> 1077966 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1077966 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1077966 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1077966 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1077966 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1077966 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1078733 T2256 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:25724__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1078733 T2256 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:25724__collection1&state=recovering&nodeName=127.0.0.1:25724_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 1078737 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1079739 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C347_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:25724_, base_url=http://127.0.0.1:25724}
[junit4:junit4] 2> 1080735 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57978/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1080735 T2304 C347 P25724 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:25724 START replicas=[http://127.0.0.1:57978/collection1/] nUpdates=100
[junit4:junit4] 2> 1080736 T2304 C347 P25724 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1080737 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1080737 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1080737 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1080737 T2304 C347 P25724 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1080738 T2304 C347 P25724 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57978/collection1/. core=collection1
[junit4:junit4] 2> 1080738 T2304 C347 P25724 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1080742 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C348 name=collection1 org.apache.solr.core.SolrCore@3213d0c0 url=http://127.0.0.1:57978/collection1 node=127.0.0.1:57978_ C348_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57978_, base_url=http://127.0.0.1:57978, leader=true}
[junit4:junit4] 2> 1080746 T2257 C348 P57978 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1080752 T2258 C348 P57978 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1080755 T2258 C348 P57978 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1080756 T2258 C348 P57978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1080757 T2258 C348 P57978 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1080757 T2258 C348 P57978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1080758 T2258 C348 P57978 oass.SolrIndexSearcher.<init> Opening Searcher@4d5cb599 realtime
[junit4:junit4] 2> 1080758 T2258 C348 P57978 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1080759 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1080760 T2304 C347 P25724 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1080760 T2304 C347 P25724 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1080762 T2259 C348 P57978 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1080762 T2259 C348 P57978 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1080763 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1080763 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1080763 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1080766 T2259 C348 P57978 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1080767 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1080768 T2304 C347 P25724 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index.20130620153116139
[junit4:junit4] 2> 1080769 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@5894cb8f lockFactory=org.apache.lucene.store.NativeFSLockFactory@750c553b) fullCopy=false
[junit4:junit4] 2> 1080771 T2259 C348 P57978 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1080772 T2304 C347 P25724 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1080773 T2304 C347 P25724 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1080774 T2304 C347 P25724 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1080775 T2304 C347 P25724 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@778c59bf lockFactory=org.apache.lucene.store.NativeFSLockFactory@4422364e)),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@778c59bf lockFactory=org.apache.lucene.store.NativeFSLockFactory@4422364e)),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1080776 T2304 C347 P25724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1080776 T2304 C347 P25724 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1080777 T2304 C347 P25724 oass.SolrIndexSearcher.<init> Opening Searcher@b036577 main
[junit4:junit4] 2> 1080778 T2303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b036577 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1080779 T2304 C347 P25724 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index.20130620153116139 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index.20130620153116139;done=true>>]
[junit4:junit4] 2> 1080779 T2304 C347 P25724 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index.20130620153116139
[junit4:junit4] 2> 1080779 T2304 C347 P25724 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index.20130620153116139
[junit4:junit4] 2> 1080780 T2304 C347 P25724 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1080780 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1080780 T2304 C347 P25724 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1080780 T2304 C347 P25724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1080782 T2304 C347 P25724 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1080973 T2231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1080974 T2231 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:25724__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25724_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25724"}
[junit4:junit4] 2> 1080988 T2266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1080988 T2250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1080988 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1080988 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1080988 T2282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1080988 T2230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1081744 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1081746 T2208 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C349 name=collection1 org.apache.solr.core.SolrCore@6d4f0d00 url=http://127.0.0.1:44754/collection1 node=127.0.0.1:44754_ C349_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:44754_, base_url=http://127.0.0.1:44754, leader=true}
[junit4:junit4] 2> 1081755 T2220 C349 P44754 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1081759 T2220 C349 P44754 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cc34d62 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4054302b)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1081760 T2220 C349 P44754 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1081761 T2220 C349 P44754 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cc34d62 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4054302b)),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cc34d62 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4054302b)),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1081761 T2220 C349 P44754 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1081762 T2220 C349 P44754 oass.SolrIndexSearcher.<init> Opening Searcher@4b61d902 main
[junit4:junit4] 2> 1081763 T2220 C349 P44754 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1081763 T2234 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b61d902 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1081764 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> C347_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25724_, base_url=http://127.0.0.1:25724}
[junit4:junit4] 2> 1081784 T2289 C347 P25724 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50758/collection1/, StdNode: http://127.0.0.1:35209/collection1/, StdNode: http://127.0.0.1:57978/collection1/, StdNode: http://127.0.0.1:25724/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1081788 T2244 C346 P50758 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1081788 T2260 C348 P57978 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1081788 T2290 C347 P25724 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1081791 T2260 C348 P57978 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7569d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c84222f)),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1081789 T2244 C346 P50758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@34d59ee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4d8d7d)),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1081792 T2260 C348 P57978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1081792 T2290 C347 P25724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@778c59bf lockFactory=org.apache.lucene.store.NativeFSLockFactory@4422364e)),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@778c59bf lockFactory=org.apache.lucene.store.NativeFSLockFactory@4422364e)),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1081793 T2260 C348 P57978 oass.SolrIndexSearcher.<init> Opening Searcher@1ea0d37b main
[junit4:junit4] 2> 1081793 T2244 C346 P50758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1081794 T2260 C348 P57978 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1081794 T2290 C347 P25724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1081795 T2268 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ea0d37b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1081795 T2244 C346 P50758 oass.SolrIndexSearcher.<init> Opening Searcher@422a5236 main
[junit4:junit4] 2> 1081796 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> C345_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35209_, base_url=http://127.0.0.1:35209}
[junit4:junit4] 2> 1081797 T2272 C345 P35209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1081796 T2290 C347 P25724 oass.SolrIndexSearcher.<init> Opening Searcher@11d88828 main
[junit4:junit4] 2> 1081798 T2272 C345 P35209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@455d4e2f lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a15d8b)),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@455d4e2f lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a15d8b)),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1081798 T2252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@422a5236 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1081797 T2244 C346 P50758 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1081799 T2303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11d88828 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1081799 T2272 C345 P35209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1081798 T2290 C347 P25724 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1081801 T2272 C345 P35209 oass.SolrIndexSearcher.<init> Opening Searcher@228fc0a5 main
[junit4:junit4] 2> 1081800 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1081802 T2272 C345 P35209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1081801 T2290 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1081803 T2284 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@228fc0a5 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1081804 T2272 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1081804 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 1081806 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1081809 T2245 C346 P50758 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1081811 T2273 C345 P35209 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1081813 T2261 C348 P57978 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1081815 T2291 C347 P25724 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1083820 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438359045213782016)} 0 2
[junit4:junit4] 2> 1083830 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438359045217976320&update.from=http://127.0.0.1:50758/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438359045217976320)} 0 3
[junit4:junit4] 2> 1083833 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438359045222170624&update.from=http://127.0.0.1:57978/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438359045222170624)} 0 2
[junit4:junit4] 2> 1083834 T2262 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438359045222170624)} 0 7
[junit4:junit4] 2> 1083835 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438359045217976320)} 0 12
[junit4:junit4] 2> 1083839 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438359045233704960)]} 0 1
[junit4:junit4] 2> 1083848 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438359045239996416)]} 0 1
[junit4:junit4] 2> 1083849 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438359045239996416)]} 0 5
[junit4:junit4] 2> 1083849 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 8
[junit4:junit4] 2> 1083853 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438359045248385024)]} 0 1
[junit4:junit4] 2> 1083860 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438359045253627904)]} 0 0
[junit4:junit4] 2> 1083861 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438359045253627904)]} 0 4
[junit4:junit4] 2> 1083862 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 7
[junit4:junit4] 2> 1083866 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438359045262016512)]} 0 1
[junit4:junit4] 2> 1083871 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1438359045264113664)]} 0 1
[junit4:junit4] 2> 1083871 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438359045264113664)]} 0 4
[junit4:junit4] 2> 1083874 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438359045271453696)]} 0 0
[junit4:junit4] 2> 1083883 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1438359045276696576)]} 0 1
[junit4:junit4] 2> 1083883 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1438359045276696576)]} 0 4
[junit4:junit4] 2> 1083884 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4] 2> 1083887 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438359045285085184)]} 0 0
[junit4:junit4] 2> 1083895 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1438359045290328064)]} 0 1
[junit4:junit4] 2> 1083895 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1438359045290328064)]} 0 4
[junit4:junit4] 2> 1083896 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4]} 0 6
[junit4:junit4] 2> 1083899 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1438359045297668096)]} 0 0
[junit4:junit4] 2> 1083907 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1438359045302910976)]} 0 0
[junit4:junit4] 2> 1083908 T2262 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1438359045302910976)]} 0 4
[junit4:junit4] 2> 1083909 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5]} 0 7
[junit4:junit4] 2> 1083912 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1438359045311299584)]} 0 0
[junit4:junit4] 2> 1083921 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1438359045316542464)]} 0 1
[junit4:junit4] 2> 1083921 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1438359045316542464)]} 0 4
[junit4:junit4] 2> 1083922 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6]} 0 7
[junit4:junit4] 2> 1083926 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1438359045324931072)]} 0 1
[junit4:junit4] 2> 1083933 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1438359045330173952)]} 0 1
[junit4:junit4] 2> 1083933 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1438359045330173952)]} 0 4
[junit4:junit4] 2> 1083934 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7]} 0 6
[junit4:junit4] 2> 1083937 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438359045337513984)]} 0 0
[junit4:junit4] 2> 1083944 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1438359045340659712)]} 0 1
[junit4:junit4] 2> 1083944 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438359045340659712)]} 0 4
[junit4:junit4] 2> 1083947 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1438359045347999744)]} 0 0
[junit4:junit4] 2> 1083956 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1438359045353242624)]} 0 1
[junit4:junit4] 2> 1083956 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1438359045353242624)]} 0 4
[junit4:junit4] 2> 1083957 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9]} 0 7
[junit4:junit4] 2> 1083960 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1438359045361631232)]} 0 0
[junit4:junit4] 2> 1083969 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1438359045366874112)]} 0 1
[junit4:junit4] 2> 1083969 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1438359045366874112)]} 0 4
[junit4:junit4] 2> 1083970 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10]} 0 7
[junit4:junit4] 2> 1083973 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1438359045375262720)]} 0 0
[junit4:junit4] 2> 1083982 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1438359045380505600)]} 0 1
[junit4:junit4] 2> 1083982 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1438359045380505600)]} 0 4
[junit4:junit4] 2> 1083983 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11]} 0 7
[junit4:junit4] 2> 1083987 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1438359045388894208)]} 0 1
[junit4:junit4] 2> 1083995 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1438359045394137088)]} 0 1
[junit4:junit4] 2> 1083995 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1438359045394137088)]} 0 4
[junit4:junit4] 2> 1083996 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12]} 0 7
[junit4:junit4] 2> 1084000 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438359045402525696)]} 0 1
[junit4:junit4] 2> 1084006 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1438359045405671424)]} 0 1
[junit4:junit4] 2> 1084006 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438359045405671424)]} 0 4
[junit4:junit4] 2> 1084010 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1438359045413011456)]} 0 1
[junit4:junit4] 2> 1084017 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1438359045418254336)]} 0 0
[junit4:junit4] 2> 1084018 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1438359045418254336)]} 0 4
[junit4:junit4] 2> 1084019 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14]} 0 7
[junit4:junit4] 2> 1084022 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1438359045426642944)]} 0 0
[junit4:junit4] 2> 1084030 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1438359045431885824)]} 0 0
[junit4:junit4] 2> 1084031 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1438359045431885824)]} 0 4
[junit4:junit4] 2> 1084032 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 1084036 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1438359045440274432)]} 0 1
[junit4:junit4] 2> 1084043 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1438359045445517312)]} 0 0
[junit4:junit4] 2> 1084044 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1438359045445517312)]} 0 4
[junit4:junit4] 2> 1084045 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16]} 0 7
[junit4:junit4] 2> 1084049 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1438359045453905920)]} 0 1
[junit4:junit4] 2> 1084057 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1438359045459148800)]} 0 1
[junit4:junit4] 2> 1084057 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1438359045459148800)]} 0 4
[junit4:junit4] 2> 1084058 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4] 2> 1084062 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1438359045467537408)]} 0 1
[junit4:junit4] 2> 1084070 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438359045472780288)]} 0 1
[junit4:junit4] 2> 1084070 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1438359045472780288)]} 0 4
[junit4:junit4] 2> 1084071 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18]} 0 7
[junit4:junit4] 2> 1084074 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1438359045481168896)]} 0 0
[junit4:junit4] 2> 1084082 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438359045486411776)]} 0 0
[junit4:junit4] 2> 1084083 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1438359045486411776)]} 0 4
[junit4:junit4] 2> 1084084 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19]} 0 7
[junit4:junit4] 2> 1084087 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1438359045494800384)]} 0 0
[junit4:junit4] 2> 1084096 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438359045500043264)]} 0 0
[junit4:junit4] 2> 1084097 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1438359045500043264)]} 0 5
[junit4:junit4] 2> 1084098 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20]} 0 8
[junit4:junit4] 2> 1084102 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1438359045509480448)]} 0 1
[junit4:junit4] 2> 1084110 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438359045514723328)]} 0 1
[junit4:junit4] 2> 1084110 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1438359045514723328)]} 0 4
[junit4:junit4] 2> 1084111 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21]} 0 7
[junit4:junit4] 2> 1084115 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1438359045523111936)]} 0 1
[junit4:junit4] 2> 1084122 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438359045528354816)]} 0 0
[junit4:junit4] 2> 1084123 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1438359045528354816)]} 0 4
[junit4:junit4] 2> 1084124 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22]} 0 7
[junit4:junit4] 2> 1084127 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438359045536743424)]} 0 0
[junit4:junit4] 2> 1084134 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438359045539889152)]} 0 1
[junit4:junit4] 2> 1084134 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438359045539889152)]} 0 4
[junit4:junit4] 2> 1084137 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1438359045547229184)]} 0 0
[junit4:junit4] 2> 1084145 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438359045552472064)]} 0 0
[junit4:junit4] 2> 1084146 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1438359045552472064)]} 0 4
[junit4:junit4] 2> 1084147 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24]} 0 7
[junit4:junit4] 2> 1084151 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1438359045560860672)]} 0 1
[junit4:junit4] 2> 1084159 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1438359045566103552)]} 0 1
[junit4:junit4] 2> 1084159 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1438359045566103552)]} 0 4
[junit4:junit4] 2> 1084160 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4] 2> 1084164 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438359045574492160)]} 0 1
[junit4:junit4] 2> 1084169 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1438359045577637888)]} 0 0
[junit4:junit4] 2> 1084170 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438359045577637888)]} 0 4
[junit4:junit4] 2> 1084174 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438359045584977920)]} 0 1
[junit4:junit4] 2> 1084180 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1438359045588123648)]} 0 1
[junit4:junit4] 2> 1084180 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438359045588123648)]} 0 4
[junit4:junit4] 2> 1084183 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1438359045595463680)]} 0 0
[junit4:junit4] 2> 1084191 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1438359045600706560)]} 0 0
[junit4:junit4] 2> 1084192 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1438359045600706560)]} 0 4
[junit4:junit4] 2> 1084193 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 1084196 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438359045609095168)]} 0 0
[junit4:junit4] 2> 1084202 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1438359045612240896)]} 0 0
[junit4:junit4] 2> 1084203 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438359045612240896)]} 0 4
[junit4:junit4] 2> 1084207 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438359045619580928)]} 0 1
[junit4:junit4] 2> 1084213 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1438359045622726656)]} 0 1
[junit4:junit4] 2> 1084213 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438359045622726656)]} 0 4
[junit4:junit4] 2> 1084216 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1438359045630066688)]} 0 0
[junit4:junit4] 2> 1084224 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1438359045635309568)]} 0 0
[junit4:junit4] 2> 1084225 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1438359045635309568)]} 0 4
[junit4:junit4] 2> 1084226 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4] 2> 1084229 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1438359045643698176)]} 0 0
[junit4:junit4] 2> 1084238 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1438359045648941056)]} 0 1
[junit4:junit4] 2> 1084238 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1438359045648941056)]} 0 4
[junit4:junit4] 2> 1084239 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32]} 0 7
[junit4:junit4] 2> 1084242 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1438359045657329664)]} 0 0
[junit4:junit4] 2> 1084251 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1438359045662572544)]} 0 1
[junit4:junit4] 2> 1084251 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1438359045662572544)]} 0 4
[junit4:junit4] 2> 1084252 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33]} 0 7
[junit4:junit4] 2> 1084256 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1438359045670961152)]} 0 1
[junit4:junit4] 2> 1084264 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1438359045676204032)]} 0 1
[junit4:junit4] 2> 1084264 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1438359045676204032)]} 0 4
[junit4:junit4] 2> 1084265 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34]} 0 7
[junit4:junit4] 2> 1084268 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1438359045684592640)]} 0 0
[junit4:junit4] 2> 1084277 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1438359045689835520)]} 0 1
[junit4:junit4] 2> 1084277 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1438359045689835520)]} 0 4
[junit4:junit4] 2> 1084278 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35]} 0 7
[junit4:junit4] 2> 1084281 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1438359045698224128)]} 0 0
[junit4:junit4] 2> 1084289 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1438359045703467008)]} 0 0
[junit4:junit4] 2> 1084290 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1438359045703467008)]} 0 4
[junit4:junit4] 2> 1084291 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36]} 0 7
[junit4:junit4] 2> 1084294 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1438359045711855616)]} 0 0
[junit4:junit4] 2> 1084303 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1438359045717098496)]} 0 1
[junit4:junit4] 2> 1084304 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1438359045717098496)]} 0 5
[junit4:junit4] 2> 1084305 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37]} 0 8
[junit4:junit4] 2> 1084309 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1438359045726535680)]} 0 1
[junit4:junit4] 2> 1084318 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1438359045731778560)]} 0 1
[junit4:junit4] 2> 1084318 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1438359045731778560)]} 0 5
[junit4:junit4] 2> 1084319 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38]} 0 8
[junit4:junit4] 2> 1084323 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1438359045741215744)]} 0 1
[junit4:junit4] 2> 1084331 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1438359045746458624)]} 0 1
[junit4:junit4] 2> 1084331 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1438359045746458624)]} 0 4
[junit4:junit4] 2> 1084332 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39]} 0 7
[junit4:junit4] 2> 1084336 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1438359045754847232)]} 0 1
[junit4:junit4] 2> 1084344 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1438359045760090112)]} 0 0
[junit4:junit4] 2> 1084344 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1438359045760090112)]} 0 4
[junit4:junit4] 2> 1084345 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40]} 0 7
[junit4:junit4] 2> 1084348 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1438359045768478720)]} 0 0
[junit4:junit4] 2> 1084356 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1438359045773721600)]} 0 0
[junit4:junit4] 2> 1084357 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1438359045773721600)]} 0 4
[junit4:junit4] 2> 1084358 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41]} 0 7
[junit4:junit4] 2> 1084361 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1438359045782110208)]} 0 0
[junit4:junit4] 2> 1084369 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1438359045787353088)]} 0 0
[junit4:junit4] 2> 1084370 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1438359045787353088)]} 0 4
[junit4:junit4] 2> 1084371 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42]} 0 7
[junit4:junit4] 2> 1084374 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1438359045795741696)]} 0 0
[junit4:junit4] 2> 1084384 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1438359045802033152)]} 0 1
[junit4:junit4] 2> 1084385 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1438359045802033152)]} 0 5
[junit4:junit4] 2> 1084385 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43]} 0 8
[junit4:junit4] 2> 1084389 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438359045810421760)]} 0 1
[junit4:junit4] 2> 1084395 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1438359045813567488)]} 0 1
[junit4:junit4] 2> 1084396 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438359045813567488)]} 0 5
[junit4:junit4] 2> 1084399 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1438359045820907520)]} 0 1
[junit4:junit4] 2> 1084409 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1438359045828247552)]} 0 1
[junit4:junit4] 2> 1084410 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[45 (1438359045828247552)]} 0 5
[junit4:junit4] 2> 1084410 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45]} 0 8
[junit4:junit4] 2> 1084414 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1438359045836636160)]} 0 1
[junit4:junit4] 2> 1084422 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1438359045841879040)]} 0 1
[junit4:junit4] 2> 1084422 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1438359045841879040)]} 0 4
[junit4:junit4] 2> 1084423 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46]} 0 7
[junit4:junit4] 2> 1084426 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1438359045850267648)]} 0 0
[junit4:junit4] 2> 1084434 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1438359045855510528)]} 0 0
[junit4:junit4] 2> 1084435 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[47 (1438359045855510528)]} 0 4
[junit4:junit4] 2> 1084436 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47]} 0 7
[junit4:junit4] 2> 1084439 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1438359045863899136)]} 0 0
[junit4:junit4] 2> 1084448 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1438359045869142016)]} 0 1
[junit4:junit4] 2> 1084448 T2242 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1438359045869142016)]} 0 4
[junit4:junit4] 2> 1084449 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48]} 0 7
[junit4:junit4] 2> 1084453 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438359045877530624)]} 0 1
[junit4:junit4] 2> 1084458 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1438359045880676352)]} 0 0
[junit4:junit4] 2> 1084459 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438359045880676352)]} 0 4
[junit4:junit4] 2> 1084463 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438359045888016384)]} 0 1
[junit4:junit4] 2> 1084468 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1438359045891162112)]} 0 0
[junit4:junit4] 2> 1084469 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438359045891162112)]} 0 4
[junit4:junit4] 2> 1084472 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1438359045898502144)]} 0 0
[junit4:junit4] 2> 1084480 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1438359045903745024)]} 0 0
[junit4:junit4] 2> 1084481 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1438359045903745024)]} 0 4
[junit4:junit4] 2> 1084482 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51]} 0 7
[junit4:junit4] 2> 1084486 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1438359045912133632)]} 0 1
[junit4:junit4] 2> 1084494 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1438359045917376512)]} 0 1
[junit4:junit4] 2> 1084494 T2240 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[52 (1438359045917376512)]} 0 4
[junit4:junit4] 2> 1084495 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52]} 0 7
[junit4:junit4] 2> 1084499 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438359045925765120)]} 0 1
[junit4:junit4] 2> 1084504 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1438359045928910848)]} 0 0
[junit4:junit4] 2> 1084505 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438359045928910848)]} 0 4
[junit4:junit4] 2> 1084508 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438359045936250880)]} 0 0
[junit4:junit4] 2> 1084515 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1438359045939396608)]} 0 1
[junit4:junit4] 2> 1084515 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438359045939396608)]} 0 4
[junit4:junit4] 2> 1084519 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438359045946736640)]} 0 1
[junit4:junit4] 2> 1084525 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1438359045949882368)]} 0 1
[junit4:junit4] 2> 1084525 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438359045949882368)]} 0 4
[junit4:junit4] 2> 1084529 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1438359045957222400)]} 0 0
[junit4:junit4] 2> 1084537 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1438359045962465280)]} 0 1
[junit4:junit4] 2> 1084537 T2260 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1438359045962465280)]} 0 4
[junit4:junit4] 2> 1084538 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56]} 0 7
[junit4:junit4] 2> 1084542 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1438359045970853888)]} 0 1
[junit4:junit4] 2> 1084551 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1438359045977145344)]} 0 1
[junit4:junit4] 2> 1084551 T2262 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1438359045977145344)]} 0 4
[junit4:junit4] 2> 1084552 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57]} 0 7
[junit4:junit4] 2> 1084556 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438359045985533952)]} 0 1
[junit4:junit4] 2> 1084563 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1438359045989728256)]} 0 1
[junit4:junit4] 2> 1084564 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438359045989728256)]} 0 5
[junit4:junit4] 2> 1084568 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438359045998116864)]} 0 1
[junit4:junit4] 2> 1084574 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1438359046001262592)]} 0 1
[junit4:junit4] 2> 1084574 T2246 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438359046001262592)]} 0 4
[junit4:junit4] 2> 1084577 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438359046008602624)]} 0 0
[junit4:junit4] 2> 1084584 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1438359046011748352)]} 0 1
[junit4:junit4] 2> 1084584 T2256 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438359046011748352)]} 0 4
[junit4:junit4] 2> 1084587 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1438359046019088384)]} 0 0
[junit4:junit4] 2> 1084596 T2274 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50758/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1438359046024331264)]} 0 1
[junit4:junit4] 2> 1084597 T2244 C346 P50758 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25724/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1438359046024331264)]} 0 5
[junit4:junit4] 2> 1084597 T2289 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61]} 0 7
[junit4:junit4] 2> 1084601 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1438359046032719872)]} 0 1
[junit4:junit4] 2> 1084609 T2292 C347 P25724 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57978/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1438359046037962752)]} 0 1
[junit4:junit4] 2> 1084609 T2258 C348 P57978 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35209/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[62 (1438359046037962752)]} 0 4
[junit4:junit4] 2> 1084610 T2275 C345 P35209 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62]} 0 7
[junit4:junit4] 2> 1084614 T2220 C349 P44754 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (14383590460463
[...truncated too long message...]
http://127.0.0.1:25724 couldn't connect to http://127.0.0.1:57978/collection1/, counting as success
[junit4:junit4] 2> 1312015 T2299 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:25724 DONE. sync succeeded
[junit4:junit4] 2> 1312015 T2299 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 1312015 T2299 oasc.ShardLeaderElectionContext.runLeaderProcess Sync was not a success but no one else is active! I am the leader
[junit4:junit4] 2> 1312016 T2299 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25724/collection1/
[junit4:junit4] 2> 1312016 T2299 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@41ffadad
[junit4:junit4] 2> 1312017 T2299 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=609,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1312018 T2299 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1312018 T2299 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1312018 T2299 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1312021 T2299 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1312022 T2299 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1312022 T2299 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index;done=false>>]
[junit4:junit4] 2> 1312022 T2299 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4/index
[junit4:junit4] 2> 1312023 T2299 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4;done=false>>]
[junit4:junit4] 2> 1312023 T2299 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371726051130/jetty4
[junit4:junit4] 2> 1312023 T2299 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1312024 T2299 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89897438487379980-127.0.0.1:25724__collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 1312024 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1312024 T2299 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1312025 T2299 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1312025 T2299 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1312025 T2299 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1312025 T2299 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, locale=fi_FI, timezone=Asia/Tehran
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=304709000,total=455081984
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReversedWildcardFilterFactory, SampleTest, TestSolr4Spatial, PluginInfoTest, TestLFUCache, SuggesterTest, TestGroupingSearch, PathHierarchyTokenizerFactoryTest, FastVectorHighlighterTest, SimplePostToolTest, TestRTGBase, ShowFileRequestHandlerTest, SolrIndexConfigTest, AliasIntegrationTest, DistributedSpellCheckComponentTest, TestStressLucene, DirectUpdateHandlerTest, ZkNodePropsTest, ConvertedLegacyTest, TestSuggestSpellingConverter, TestFunctionQuery, TestRandomDVFaceting, TestDefaultSimilarityFactory, SpellingQueryConverterTest, TestExtendedDismaxParser, TestSolrIndexConfig, UpdateParamsTest, TestSolrDeletionPolicy1, IndexReaderFactoryTest, NotRequiredUniqueKeyTest, TestPerFieldSimilarity, TestDistributedSearch, TestJmxIntegration, ShardRoutingCustomTest, IndexBasedSpellCheckerTest, FieldMutatingUpdateProcessorTest, QueryResultKeyTest, DirectSolrSpellCheckerTest, RegexBoostProcessorTest, MultiTermTest, TestComponentsName, TestJmxMonitoredMap, TestFastOutputStream, TestCoreContainer, TestManagedSchema, TestSolrDeletionPolicy2, OverseerTest, TestLazyCores, CacheHeaderTest, TermsComponentTest, TestAnalyzedSuggestions, FileBasedSpellCheckerTest, SignatureUpdateProcessorFactoryTest, TestHashPartitioner, TestClassNameShortening, RequestHandlersTest, BadIndexSchemaTest, SolrCmdDistributorTest, DocumentBuilderTest, DOMUtilTest, TestCloudManagedSchemaAddField, BasicDistributedZk2Test, SuggesterWFSTTest, TestCoreDiscovery, NumericFieldsTest, TestLuceneMatchVersion, TestConfig, TestNumberUtils, ChaosMonkeyNothingIsSafeTest, TestIndexSearcher, BasicZkTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 256.79s, 1 test, 1 error <<< FAILURES!
[...truncated 466 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1246: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:890: There were test failures: 298 suites, 1255 tests, 1 error, 13 ignored (7 assumptions)
Total time: 54 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1344 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1344/
3 tests failed.
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<206> but was:<34>
Stack Trace:
java.lang.AssertionError: expected:<206> but was:<34>
at __randomizedtesting.SeedInfo.seed([E26826E5A65C9485:638EA8FDD103F4B9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:46917/d/pl/collection1lastClient and got 263 from http://127.0.0.1:46930/d/pl/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:46917/d/pl/collection1lastClient and got 263 from http://127.0.0.1:46930/d/pl/collection1
at __randomizedtesting.SeedInfo.seed([E26826E5A65C9485:638EA8FDD103F4B9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:50770 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:50770 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([E26826E5A65C9485:638EA8FDD103F4B9]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9667 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4:junit4] 2> 1061784 T2627 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1061790 T2627 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1371691421668
[junit4:junit4] 2> 1061791 T2627 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1061792 T2628 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1061893 T2627 oasc.ZkTestServer.run start zk server on port:41797
[junit4:junit4] 2> 1061894 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1061898 T2634 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47c997e0 name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1061898 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1061899 T2627 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1061915 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1061917 T2636 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74b130a5 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1061918 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1061918 T2627 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1061929 T2627 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1061932 T2627 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1061934 T2627 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1061937 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1061938 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1061951 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1061952 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1062066 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1062067 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1062076 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1062077 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1062080 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1062081 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1062085 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1062085 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1062088 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1062089 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1062092 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1062093 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1062096 T2627 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1062097 T2627 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1062361 T2627 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1062366 T2627 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41800
[junit4:junit4] 2> 1062367 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1062368 T2627 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1062368 T2627 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985
[junit4:junit4] 2> 1062369 T2627 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/solr.xml
[junit4:junit4] 2> 1062369 T2627 oasc.CoreContainer.<init> New CoreContainer 957273747
[junit4:junit4] 2> 1062370 T2627 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/'
[junit4:junit4] 2> 1062370 T2627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/'
[junit4:junit4] 2> 1062502 T2627 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1062502 T2627 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1062503 T2627 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1062504 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1062504 T2627 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1062505 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1062505 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1062506 T2627 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1062506 T2627 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1062507 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1062525 T2627 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1062525 T2627 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41797/solr
[junit4:junit4] 2> 1062526 T2627 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1062527 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1062530 T2647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53aabe1c name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1062530 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1062533 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1062545 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1062547 T2649 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14124673 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1062548 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1062550 T2627 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1062560 T2627 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1062564 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1062571 T2627 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41800_
[junit4:junit4] 2> 1062572 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41800_
[junit4:junit4] 2> 1062576 T2627 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1062595 T2627 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1062598 T2627 oasc.Overseer.start Overseer (id=89895169010892803-127.0.0.1:41800_-n_0000000000) starting
[junit4:junit4] 2> 1062608 T2627 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1062620 T2651 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1062621 T2627 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1062624 T2627 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1062626 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1062628 T2650 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1062632 T2652 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1062633 T2652 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1064133 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1064134 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41800_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41800"}
[junit4:junit4] 2> 1064134 T2650 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1064134 T2650 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1064153 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1064637 T2652 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/collection1
[junit4:junit4] 2> 1064637 T2652 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1064638 T2652 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1064638 T2652 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1064640 T2652 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/collection1/'
[junit4:junit4] 2> 1064641 T2652 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1064642 T2652 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/collection1/lib/README' to classloader
[junit4:junit4] 2> 1064696 T2652 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1064758 T2652 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1064859 T2652 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1064865 T2652 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1065483 T2652 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1065487 T2652 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1065490 T2652 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1065495 T2652 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1065521 T2652 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1065521 T2652 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371691421985/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data/
[junit4:junit4] 2> 1065521 T2652 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a003082
[junit4:junit4] 2> 1065522 T2652 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1065523 T2652 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data
[junit4:junit4] 2> 1065523 T2652 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data/index/
[junit4:junit4] 2> 1065523 T2652 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1065524 T2652 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data/index
[junit4:junit4] 2> 1065529 T2652 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57d67785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f8a909d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1065529 T2652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1065532 T2652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1065533 T2652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1065534 T2652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1065534 T2652 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1065535 T2652 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1065535 T2652 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1065536 T2652 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1065536 T2652 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1065537 T2652 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1065554 T2652 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1065561 T2652 oass.SolrIndexSearcher.<init> Opening Searcher@7419305a main
[junit4:junit4] 2> 1065562 T2652 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/control/data/tlog
[junit4:junit4] 2> 1065563 T2652 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1065563 T2652 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1065567 T2653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7419305a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1065569 T2652 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1065570 T2652 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41800 collection:control_collection shard:shard1
[junit4:junit4] 2> 1065570 T2652 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1065687 T2652 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1065691 T2652 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1065691 T2652 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1065691 T2652 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41800/collection1/
[junit4:junit4] 2> 1065692 T2652 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1065692 T2652 oasc.SyncStrategy.syncToMe http://127.0.0.1:41800/collection1/ has no replicas
[junit4:junit4] 2> 1065692 T2652 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41800/collection1/
[junit4:junit4] 2> 1065692 T2652 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1067191 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1067213 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1067265 T2652 oasc.ZkController.register We are http://127.0.0.1:41800/collection1/ and leader is http://127.0.0.1:41800/collection1/
[junit4:junit4] 2> 1067265 T2652 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41800
[junit4:junit4] 2> 1067265 T2652 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1067265 T2652 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1067266 T2652 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1067268 T2652 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1067269 T2627 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1067269 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1067270 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1067288 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1067289 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1067291 T2656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78580c62 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1067292 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1067293 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1067295 T2627 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1067558 T2627 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1067560 T2627 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50770
[junit4:junit4] 2> 1067561 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1067562 T2627 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1067562 T2627 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173
[junit4:junit4] 2> 1067563 T2627 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/solr.xml
[junit4:junit4] 2> 1067563 T2627 oasc.CoreContainer.<init> New CoreContainer 1335517860
[junit4:junit4] 2> 1067564 T2627 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/'
[junit4:junit4] 2> 1067565 T2627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/'
[junit4:junit4] 2> 1067697 T2627 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1067698 T2627 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1067698 T2627 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1067699 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1067699 T2627 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1067700 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1067700 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1067701 T2627 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1067701 T2627 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1067702 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1067718 T2627 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1067718 T2627 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41797/solr
[junit4:junit4] 2> 1067719 T2627 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1067720 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1067723 T2667 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2703c450 name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1067724 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1067740 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1067750 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1067752 T2669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d5d0af5 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1067753 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1067763 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1068724 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1068725 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41800__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41800_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41800"}
[junit4:junit4] 2> 1068740 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1068740 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1068740 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1068767 T2627 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50770_
[junit4:junit4] 2> 1068769 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50770_
[junit4:junit4] 2> 1068778 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1068778 T2656 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1068779 T2649 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1068779 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1068779 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1068780 T2669 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1068786 T2670 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1068786 T2670 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1070270 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1070271 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50770_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50770"}
[junit4:junit4] 2> 1070271 T2650 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1070271 T2650 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1070285 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1070285 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1070285 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1070790 T2670 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/collection1
[junit4:junit4] 2> 1070790 T2670 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1070791 T2670 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1070791 T2670 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1070793 T2670 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/collection1/'
[junit4:junit4] 2> 1070795 T2670 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/collection1/lib/README' to classloader
[junit4:junit4] 2> 1070795 T2670 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1070850 T2670 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1070912 T2670 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1071013 T2670 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1071019 T2670 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1071601 T2670 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1071605 T2670 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1071607 T2670 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1071612 T2670 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1071637 T2670 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1071637 T2670 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371691427173/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1/
[junit4:junit4] 2> 1071637 T2670 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a003082
[junit4:junit4] 2> 1071638 T2670 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1071639 T2670 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1
[junit4:junit4] 2> 1071639 T2670 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1/index/
[junit4:junit4] 2> 1071639 T2670 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1071640 T2670 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1/index
[junit4:junit4] 2> 1071644 T2670 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1071644 T2670 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1071648 T2670 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1071648 T2670 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1071649 T2670 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1071650 T2670 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1071650 T2670 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1071651 T2670 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1071651 T2670 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1071652 T2670 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1071652 T2670 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1071667 T2670 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1071674 T2670 oass.SolrIndexSearcher.<init> Opening Searcher@5548a3b5 main
[junit4:junit4] 2> 1071675 T2670 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty1/tlog
[junit4:junit4] 2> 1071676 T2670 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1071676 T2670 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1071681 T2671 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5548a3b5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1071684 T2670 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1071684 T2670 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50770 collection:collection1 shard:shard1
[junit4:junit4] 2> 1071685 T2670 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1071706 T2670 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1071715 T2670 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1071716 T2670 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1071716 T2670 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50770/collection1/
[junit4:junit4] 2> 1071716 T2670 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1071717 T2670 oasc.SyncStrategy.syncToMe http://127.0.0.1:50770/collection1/ has no replicas
[junit4:junit4] 2> 1071717 T2670 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50770/collection1/
[junit4:junit4] 2> 1071717 T2670 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1071790 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1071806 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1071806 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1071806 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1071826 T2670 oasc.ZkController.register We are http://127.0.0.1:50770/collection1/ and leader is http://127.0.0.1:50770/collection1/
[junit4:junit4] 2> 1071826 T2670 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50770
[junit4:junit4] 2> 1071826 T2670 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1071826 T2670 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1071827 T2670 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1071828 T2670 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1071829 T2627 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1071830 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1071831 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1072109 T2627 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1072112 T2627 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56319
[junit4:junit4] 2> 1072113 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1072114 T2627 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1072114 T2627 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726
[junit4:junit4] 2> 1072115 T2627 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/solr.xml
[junit4:junit4] 2> 1072115 T2627 oasc.CoreContainer.<init> New CoreContainer 1721785338
[junit4:junit4] 2> 1072116 T2627 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/'
[junit4:junit4] 2> 1072117 T2627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/'
[junit4:junit4] 2> 1072250 T2627 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1072251 T2627 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1072252 T2627 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1072252 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1072253 T2627 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1072253 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1072254 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1072254 T2627 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1072255 T2627 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1072256 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1072273 T2627 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1072274 T2627 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41797/solr
[junit4:junit4] 2> 1072275 T2627 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1072276 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1072279 T2683 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47ff4cec name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1072280 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1072294 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1072305 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1072307 T2685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c3d01d6 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1072308 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1072319 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1073317 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1073318 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50770__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50770_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50770"}
[junit4:junit4] 2> 1073322 T2627 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56319_
[junit4:junit4] 2> 1073332 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073332 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073332 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073332 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073333 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56319_
[junit4:junit4] 2> 1073342 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073342 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1073343 T2656 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1073343 T2649 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1073343 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1073343 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1073344 T2685 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1073345 T2669 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1073350 T2686 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1073351 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1074837 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1074838 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56319_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56319"}
[junit4:junit4] 2> 1074839 T2650 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1074839 T2650 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1074880 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1074880 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1074880 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1074880 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1075354 T2686 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/collection1
[junit4:junit4] 2> 1075354 T2686 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1075355 T2686 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1075355 T2686 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1075357 T2686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/collection1/'
[junit4:junit4] 2> 1075358 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1075359 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/collection1/lib/README' to classloader
[junit4:junit4] 2> 1075417 T2686 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1075481 T2686 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1075582 T2686 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1075588 T2686 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1076156 T2686 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1076160 T2686 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1076163 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1076168 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1076193 T2686 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1076193 T2686 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371691431726/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2/
[junit4:junit4] 2> 1076194 T2686 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a003082
[junit4:junit4] 2> 1076194 T2686 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1076195 T2686 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2
[junit4:junit4] 2> 1076195 T2686 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2/index/
[junit4:junit4] 2> 1076196 T2686 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1076197 T2686 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2/index
[junit4:junit4] 2> 1076201 T2686 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1076201 T2686 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1076205 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1076205 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1076206 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1076207 T2686 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1076208 T2686 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1076208 T2686 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1076208 T2686 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1076209 T2686 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1076210 T2686 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1076226 T2686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1076234 T2686 oass.SolrIndexSearcher.<init> Opening Searcher@2067d1d0 main
[junit4:junit4] 2> 1076234 T2686 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty2/tlog
[junit4:junit4] 2> 1076235 T2686 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1076236 T2686 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1076240 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2067d1d0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1076243 T2686 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1076243 T2686 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56319 collection:collection1 shard:shard2
[junit4:junit4] 2> 1076244 T2686 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1076267 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1076276 T2686 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1076276 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1076276 T2686 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56319/collection1/
[junit4:junit4] 2> 1076276 T2686 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1076277 T2686 oasc.SyncStrategy.syncToMe http://127.0.0.1:56319/collection1/ has no replicas
[junit4:junit4] 2> 1076277 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56319/collection1/
[junit4:junit4] 2> 1076277 T2686 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1076392 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1076408 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1076409 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1076408 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1076408 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1076436 T2686 oasc.ZkController.register We are http://127.0.0.1:56319/collection1/ and leader is http://127.0.0.1:56319/collection1/
[junit4:junit4] 2> 1076436 T2686 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56319
[junit4:junit4] 2> 1076436 T2686 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1076436 T2686 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1076437 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1076438 T2686 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1076439 T2627 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1076440 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1076441 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1076725 T2627 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1076728 T2627 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16528
[junit4:junit4] 2> 1076728 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1076729 T2627 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1076730 T2627 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336
[junit4:junit4] 2> 1076730 T2627 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/solr.xml
[junit4:junit4] 2> 1076731 T2627 oasc.CoreContainer.<init> New CoreContainer 2056897736
[junit4:junit4] 2> 1076731 T2627 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/'
[junit4:junit4] 2> 1076732 T2627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/'
[junit4:junit4] 2> 1076865 T2627 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1076866 T2627 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1076867 T2627 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1076867 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1076868 T2627 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1076869 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1076869 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1076870 T2627 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1076870 T2627 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1076871 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1076889 T2627 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1076889 T2627 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41797/solr
[junit4:junit4] 2> 1076890 T2627 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1076891 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1076894 T2699 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fabe6d7 name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1076895 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1076907 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1076920 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1076922 T2701 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d6478f4 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1076922 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1076934 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1077914 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1077915 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:56319__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56319_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56319"}
[junit4:junit4] 2> 1077932 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077932 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077932 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077932 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077932 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077938 T2627 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16528_
[junit4:junit4] 2> 1077940 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16528_
[junit4:junit4] 2> 1077942 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077943 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1077943 T2649 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1077943 T2656 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1077943 T2701 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1077944 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1077943 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1077944 T2685 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1077944 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1077951 T2669 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1077956 T2702 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1077957 T2702 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1079437 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1079438 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16528"}
[junit4:junit4] 2> 1079439 T2650 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1079439 T2650 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1079444 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1079444 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1079444 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1079444 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1079444 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1079960 T2702 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/collection1
[junit4:junit4] 2> 1079960 T2702 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1079961 T2702 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1079961 T2702 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1079963 T2702 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/collection1/'
[junit4:junit4] 2> 1079964 T2702 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1079965 T2702 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/collection1/lib/README' to classloader
[junit4:junit4] 2> 1080042 T2702 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1080103 T2702 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1080205 T2702 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1080211 T2702 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1080781 T2702 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1080785 T2702 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1080788 T2702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1080793 T2702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1080818 T2702 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1080819 T2702 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371691436336/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/
[junit4:junit4] 2> 1080819 T2702 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a003082
[junit4:junit4] 2> 1080820 T2702 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1080821 T2702 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3
[junit4:junit4] 2> 1080821 T2702 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index/
[junit4:junit4] 2> 1080821 T2702 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1080823 T2702 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index
[junit4:junit4] 2> 1080827 T2702 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54f759b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3517bc60),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1080827 T2702 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1080831 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1080831 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1080832 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1080833 T2702 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1080834 T2702 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1080834 T2702 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1080834 T2702 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1080835 T2702 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1080836 T2702 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1080853 T2702 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1080860 T2702 oass.SolrIndexSearcher.<init> Opening Searcher@7c508417 main
[junit4:junit4] 2> 1080861 T2702 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/tlog
[junit4:junit4] 2> 1080862 T2702 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1080862 T2702 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1080867 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7c508417 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1080870 T2702 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1080870 T2702 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16528 collection:collection1 shard:shard1
[junit4:junit4] 2> 1080877 T2702 oasc.ZkController.register We are http://127.0.0.1:16528/collection1/ and leader is http://127.0.0.1:50770/collection1/
[junit4:junit4] 2> 1080877 T2702 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16528
[junit4:junit4] 2> 1080877 T2702 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1080878 T2702 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C479 name=collection1 org.apache.solr.core.SolrCore@70ba831d url=http://127.0.0.1:16528/collection1 node=127.0.0.1:16528_ C479_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:16528_, base_url=http://127.0.0.1:16528}
[junit4:junit4] 2> 1080878 T2704 C479 P16528 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1080878 T2702 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1080879 T2704 C479 P16528 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1080879 T2704 C479 P16528 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1080879 T2704 C479 P16528 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1080880 T2627 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1080881 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1080881 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1080886 T2704 C479 P16528 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1080905 T2659 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:16528__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1080956 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1080957 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:16528__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16528"}
[junit4:junit4] 2> 1080978 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1080978 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1080978 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1080978 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1080978 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1081164 T2627 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1081167 T2627 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:60420
[junit4:junit4] 2> 1081168 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1081168 T2627 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1081169 T2627 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780
[junit4:junit4] 2> 1081169 T2627 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/solr.xml
[junit4:junit4] 2> 1081170 T2627 oasc.CoreContainer.<init> New CoreContainer 1020621521
[junit4:junit4] 2> 1081171 T2627 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/'
[junit4:junit4] 2> 1081171 T2627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/'
[junit4:junit4] 2> 1081302 T2627 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1081303 T2627 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1081304 T2627 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1081304 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1081305 T2627 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1081306 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1081306 T2627 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1081307 T2627 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1081307 T2627 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1081308 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1081325 T2627 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1081326 T2627 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41797/solr
[junit4:junit4] 2> 1081326 T2627 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1081328 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1081330 T2716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38b4d3eb name:ZooKeeperConnection Watcher:127.0.0.1:41797 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1081332 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1081343 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1081355 T2627 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1081357 T2718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5606ff11 name:ZooKeeperConnection Watcher:127.0.0.1:41797/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1081358 T2627 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1081362 T2627 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1081907 T2659 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:16528__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1081907 T2659 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:16528__collection1&state=recovering&nodeName=127.0.0.1:16528_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 1082366 T2627 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60420_
[junit4:junit4] 2> 1082368 T2627 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60420_
[junit4:junit4] 2> 1082372 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1082372 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1082372 T2656 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082372 T2649 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082372 T2701 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082372 T2718 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082373 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082373 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082372 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082373 T2685 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082383 T2669 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1082389 T2719 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1082389 T2719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1082483 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1082484 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60420"}
[junit4:junit4] 2> 1082484 T2650 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1082485 T2650 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1082498 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082498 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082498 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082498 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082498 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1082498 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1083392 T2719 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/collection1
[junit4:junit4] 2> 1083392 T2719 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1083393 T2719 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1083393 T2719 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1083395 T2719 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/collection1/'
[junit4:junit4] 2> 1083396 T2719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/collection1/lib/README' to classloader
[junit4:junit4] 2> 1083397 T2719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1083452 T2719 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1083514 T2719 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1083615 T2719 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1083621 T2719 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C480 name=collection1 org.apache.solr.core.SolrCore@70ba831d url=http://127.0.0.1:16528/collection1 node=127.0.0.1:16528_ C480_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:16528_, base_url=http://127.0.0.1:16528}
[junit4:junit4] 2> 1083909 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50770/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1083909 T2704 C480 P16528 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:16528 START replicas=[http://127.0.0.1:50770/collection1/] nUpdates=100
[junit4:junit4] 2> 1083910 T2704 C480 P16528 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1083911 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1083911 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1083911 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1083912 T2704 C480 P16528 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1083912 T2704 C480 P16528 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50770/collection1/. core=collection1
[junit4:junit4] 2> 1083912 T2704 C480 P16528 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C481 name=collection1 org.apache.solr.core.SolrCore@552695d1 url=http://127.0.0.1:50770/collection1 node=127.0.0.1:50770_ C481_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50770_, base_url=http://127.0.0.1:50770, leader=true}
[junit4:junit4] 2> 1083920 T2660 C481 P50770 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1083924 T2661 C481 P50770 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1083927 T2661 C481 P50770 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1083928 T2661 C481 P50770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1083929 T2661 C481 P50770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1083929 T2661 C481 P50770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1083930 T2661 C481 P50770 oass.SolrIndexSearcher.<init> Opening Searcher@2e9bd79c realtime
[junit4:junit4] 2> 1083930 T2661 C481 P50770 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1083931 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1083932 T2704 C480 P16528 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1083932 T2704 C480 P16528 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1083935 T2662 C481 P50770 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1083935 T2662 C481 P50770 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1083936 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1083936 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1083936 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1083938 T2662 C481 P50770 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1083939 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1083940 T2704 C480 P16528 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index.20130620032403816
[junit4:junit4] 2> 1083940 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5e734e03 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33aa2772)) fullCopy=false
[junit4:junit4] 2> 1083943 T2662 C481 P50770 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1083944 T2704 C480 P16528 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1083945 T2704 C480 P16528 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1083945 T2704 C480 P16528 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1083947 T2704 C480 P16528 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54f759b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3517bc60),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54f759b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3517bc60),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1083948 T2704 C480 P16528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1083948 T2704 C480 P16528 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1083949 T2704 C480 P16528 oass.SolrIndexSearcher.<init> Opening Searcher@58251437 main
[junit4:junit4] 2> 1083950 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58251437 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1083951 T2704 C480 P16528 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index.20130620032403816 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index.20130620032403816;done=true>>]
[junit4:junit4] 2> 1083951 T2704 C480 P16528 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index.20130620032403816
[junit4:junit4] 2> 1083951 T2704 C480 P16528 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty3/index.20130620032403816
[junit4:junit4] 2> 1083951 T2704 C480 P16528 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1083952 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1083952 T2704 C480 P16528 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1083952 T2704 C480 P16528 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1083954 T2704 C480 P16528 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1084002 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1084003 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:16528__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16528"}
[junit4:junit4] 2> 1084016 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084017 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084017 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084017 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084017 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084017 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1084222 T2719 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1084226 T2719 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1084229 T2719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1084234 T2719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1084259 T2719 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1084259 T2719 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371691440780/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/
[junit4:junit4] 2> 1084260 T2719 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a003082
[junit4:junit4] 2> 1084260 T2719 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1084261 T2719 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4
[junit4:junit4] 2> 1084261 T2719 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index/
[junit4:junit4] 2> 1084262 T2719 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1084262 T2719 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index
[junit4:junit4] 2> 1084266 T2719 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@467fc3ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b794627),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1084267 T2719 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1084270 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1084271 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1084272 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1084273 T2719 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1084273 T2719 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1084274 T2719 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1084274 T2719 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1084275 T2719 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1084276 T2719 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1084293 T2719 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1084300 T2719 oass.SolrIndexSearcher.<init> Opening Searcher@2c77b450 main
[junit4:junit4] 2> 1084301 T2719 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/tlog
[junit4:junit4] 2> 1084302 T2719 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1084302 T2719 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1084307 T2722 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c77b450 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1084310 T2719 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1084310 T2719 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60420 collection:collection1 shard:shard2
[junit4:junit4] 2> 1084313 T2719 oasc.ZkController.register We are http://127.0.0.1:60420/collection1/ and leader is http://127.0.0.1:56319/collection1/
[junit4:junit4] 2> 1084313 T2719 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60420
[junit4:junit4] 2> 1084314 T2719 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1084314 T2719 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C482 name=collection1 org.apache.solr.core.SolrCore@430ea6e2 url=http://127.0.0.1:60420/collection1 node=127.0.0.1:60420_ C482_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:60420_, base_url=http://127.0.0.1:60420}
[junit4:junit4] 2> 1084314 T2723 C482 P60420 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1084315 T2719 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1084315 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1084315 T2723 C482 P60420 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1084316 T2723 C482 P60420 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1084316 T2627 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1084317 T2627 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1084317 T2723 C482 P60420 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1084317 T2627 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1084336 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1084337 T2675 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:60420__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1084337 T2627 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1084337 T2627 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1084338 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1085340 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1085522 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1085523 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60420__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60420"}
[junit4:junit4] 2> 1085564 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1085564 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1085564 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1085564 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1085564 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1085564 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1086339 T2675 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:60420__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1086339 T2675 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:60420__collection1&state=recovering&nodeName=127.0.0.1:60420_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1086342 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1087344 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C482_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:60420_, base_url=http://127.0.0.1:60420}
[junit4:junit4] 2> 1088341 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56319/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1088341 T2723 C482 P60420 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60420 START replicas=[http://127.0.0.1:56319/collection1/] nUpdates=100
[junit4:junit4] 2> 1088342 T2723 C482 P60420 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1088342 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1088342 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1088342 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1088343 T2723 C482 P60420 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1088343 T2723 C482 P60420 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56319/collection1/. core=collection1
[junit4:junit4] 2> 1088343 T2723 C482 P60420 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1088346 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C483 name=collection1 org.apache.solr.core.SolrCore@333a5639 url=http://127.0.0.1:56319/collection1 node=127.0.0.1:56319_ C483_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56319_, base_url=http://127.0.0.1:56319, leader=true}
[junit4:junit4] 2> 1088350 T2676 C483 P56319 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1088354 T2677 C483 P56319 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1088358 T2677 C483 P56319 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1088359 T2677 C483 P56319 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1088359 T2677 C483 P56319 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1088360 T2677 C483 P56319 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1088361 T2677 C483 P56319 oass.SolrIndexSearcher.<init> Opening Searcher@20ec53dc realtime
[junit4:junit4] 2> 1088361 T2677 C483 P56319 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1088361 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1088362 T2723 C482 P60420 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1088363 T2723 C482 P60420 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1088364 T2678 C483 P56319 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1088365 T2678 C483 P56319 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1088366 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1088366 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1088366 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1088368 T2678 C483 P56319 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1088369 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1088370 T2723 C482 P60420 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index.20130620032408246
[junit4:junit4] 2> 1088370 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2ac34b87 lockFactory=org.apache.lucene.store.NativeFSLockFactory@341a8f09)) fullCopy=false
[junit4:junit4] 2> 1088373 T2678 C483 P56319 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1088374 T2723 C482 P60420 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1088375 T2723 C482 P60420 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1088375 T2723 C482 P60420 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1088377 T2723 C482 P60420 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@467fc3ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b794627),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@467fc3ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b794627),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1088377 T2723 C482 P60420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1088378 T2723 C482 P60420 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1088378 T2723 C482 P60420 oass.SolrIndexSearcher.<init> Opening Searcher@35681d54 main
[junit4:junit4] 2> 1088379 T2722 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35681d54 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1088380 T2723 C482 P60420 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index.20130620032408246 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index.20130620032408246;done=true>>]
[junit4:junit4] 2> 1088380 T2723 C482 P60420 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index.20130620032408246
[junit4:junit4] 2> 1088380 T2723 C482 P60420 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371691421667/jetty4/index.20130620032408246
[junit4:junit4] 2> 1088381 T2723 C482 P60420 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1088381 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1088381 T2723 C482 P60420 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1088381 T2723 C482 P60420 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1088383 T2723 C482 P60420 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1088638 T2650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1088639 T2650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60420__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60420"}
[junit4:junit4] 2> 1088654 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1088654 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1088654 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1088654 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1088654 T2669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1088654 T2656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1089348 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1089350 T2627 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C484 name=collection1 org.apache.solr.core.SolrCore@308434d3 url=http://127.0.0.1:41800/collection1 node=127.0.0.1:41800_ C484_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:41800_, base_url=http://127.0.0.1:41800, leader=true}
[junit4:junit4] 2> 1089358 T2639 C484 P41800 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1089362 T2639 C484 P41800 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57d67785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f8a909d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1089363 T2639 C484 P41800 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1089363 T2639 C484 P41800 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57d67785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f8a909d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57d67785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f8a909d),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1089364 T2639 C484 P41800 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1089364 T2639 C484 P41800 oass.SolrIndexSearcher.<init> Opening Searcher@1adef20c main
[junit4:junit4] 2> 1089365 T2639 C484 P41800 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1089366 T2653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1adef20c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1089367 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 1089377 T2663 C481 P50770 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50770/collection1/, StdNode: http://127.0.0.1:16528/collection1/, StdNode: http://127.0.0.1:56319/collection1/, StdNode: http://127.0.0.1:60420/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1089381 T2664 C481 P50770 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1089381 T2679 C483 P56319 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1089381 T2664 C481 P50770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16bed7e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@276d4f72),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1089382 T2679 C483 P56319 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c925395 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d4ab3c5),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1089382 T2664 C481 P50770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1089383 T2679 C483 P56319 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1089383 T2664 C481 P50770 oass.SolrIndexSearcher.<init> Opening Searcher@7a4cf31d main
[junit4:junit4] 2> 1089384 T2679 C483 P56319 oass.SolrIndexSearcher.<init> Opening Searcher@2bd9bfa8 main
[junit4:junit4] 2> 1089384 T2664 C481 P50770 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1089385 T2679 C483 P56319 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1089385 T2671 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a4cf31d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1089385 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bd9bfa8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1089386 T2664 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1089386 T2679 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> C482_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60420_, base_url=http://127.0.0.1:60420}
[junit4:junit4] 2> 1089389 T2708 C482 P60420 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C480_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:16528_, base_url=http://127.0.0.1:16528}
[junit4:junit4] 2> 1089389 T2691 C480 P16528 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1089389 T2708 C482 P60420 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@467fc3ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b794627),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@467fc3ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b794627),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1089390 T2691 C480 P16528 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54f759b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3517bc60),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54f759b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3517bc60),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1089390 T2708 C482 P60420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1089391 T2691 C480 P16528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1089391 T2708 C482 P60420 oass.SolrIndexSearcher.<init> Opening Searcher@6e6e9d4f main
[junit4:junit4] 2> 1089392 T2691 C480 P16528 oass.SolrIndexSearcher.<init> Opening Searcher@7994035a main
[junit4:junit4] 2> 1089392 T2708 C482 P60420 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1089393 T2691 C480 P16528 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1089393 T2722 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e6e9d4f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1089394 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7994035a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1089394 T2708 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1089394 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1089395 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1089397 T2627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1089399 T2665 C481 P50770 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1089402 T2692 C480 P16528 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1089403 T2680 C483 P56319 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1089405 T2709 C482 P60420 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1091411 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438322735224717312)} 0 2
[junit4:junit4] 2> 1091419 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438322735228911616&update.from=http://127.0.0.1:50770/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438322735228911616)} 0 1
[junit4:junit4] 2> 1091424 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438322735233105920&update.from=http://127.0.0.1:56319/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438322735233105920)} 0 2
[junit4:junit4] 2> 1091424 T2679 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438322735233105920)} 0 6
[junit4:junit4] 2> 1091425 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438322735228911616)} 0 11
[junit4:junit4] 2> 1091429 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0 (1438322735243591680)]} 0 1
[junit4:junit4] 2> 1091437 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!0 (1438322735248834560)]} 0 1
[junit4:junit4] 2> 1091437 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!0 (1438322735248834560)]} 0 4
[junit4:junit4] 2> 1091438 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0]} 0 8
[junit4:junit4] 2> 1091441 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1 (1438322735257223168)]} 0 0
[junit4:junit4] 2> 1091449 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!1 (1438322735261417472)]} 0 1
[junit4:junit4] 2> 1091449 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!1 (1438322735261417472)]} 0 4
[junit4:junit4] 2> 1091450 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1]} 0 6
[junit4:junit4] 2> 1091453 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2 (1438322735269806080)]} 0 0
[junit4:junit4] 2> 1091461 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!2 (1438322735275048960)]} 0 0
[junit4:junit4] 2> 1091462 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!2 (1438322735275048960)]} 0 4
[junit4:junit4] 2> 1091463 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2]} 0 7
[junit4:junit4] 2> 1091465 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3 (1438322735282388992)]} 0 0
[junit4:junit4] 2> 1091471 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!3 (1438322735286583296)]} 0 0
[junit4:junit4] 2> 1091472 T2675 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!3 (1438322735286583296)]} 0 3
[junit4:junit4] 2> 1091473 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3]} 0 6
[junit4:junit4] 2> 1091475 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4 (1438322735292874752)]} 0 0
[junit4:junit4] 2> 1091483 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!4 (1438322735298117632)]} 0 0
[junit4:junit4] 2> 1091484 T2675 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!4 (1438322735298117632)]} 0 4
[junit4:junit4] 2> 1091485 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4]} 0 7
[junit4:junit4] 2> 1091487 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5 (1438322735305457664)]} 0 0
[junit4:junit4] 2> 1091495 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!5 (1438322735310700544)]} 0 0
[junit4:junit4] 2> 1091496 T2675 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!5 (1438322735310700544)]} 0 4
[junit4:junit4] 2> 1091497 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5]} 0 7
[junit4:junit4] 2> 1091499 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438322735318040576)]} 0 0
[junit4:junit4] 2> 1091505 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!6 (1438322735321186304)]} 0 0
[junit4:junit4] 2> 1091506 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438322735321186304)]} 0 4
[junit4:junit4] 2> 1091509 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7 (1438322735328526336)]} 0 0
[junit4:junit4] 2> 1091517 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!7 (1438322735333769216)]} 0 0
[junit4:junit4] 2> 1091518 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!7 (1438322735333769216)]} 0 4
[junit4:junit4] 2> 1091519 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7]} 0 7
[junit4:junit4] 2> 1091522 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8 (1438322735342157824)]} 0 0
[junit4:junit4] 2> 1091530 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!8 (1438322735347400704)]} 0 0
[junit4:junit4] 2> 1091531 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!8 (1438322735347400704)]} 0 4
[junit4:junit4] 2> 1091532 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8]} 0 7
[junit4:junit4] 2> 1091534 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9 (1438322735354740736)]} 0 0
[junit4:junit4] 2> 1091539 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!9 (1438322735357886464)]} 0 0
[junit4:junit4] 2> 1091540 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9 (1438322735357886464)]} 0 3
[junit4:junit4] 2> 1091543 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10 (1438322735364177920)]} 0 0
[junit4:junit4] 2> 1091551 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!10 (1438322735369420800)]} 0 0
[junit4:junit4] 2> 1091552 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!10 (1438322735369420800)]} 0 4
[junit4:junit4] 2> 1091553 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10]} 0 7
[junit4:junit4] 2> 1091555 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11 (1438322735376760832)]} 0 0
[junit4:junit4] 2> 1091563 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!11 (1438322735382003712)]} 0 0
[junit4:junit4] 2> 1091564 T2679 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!11 (1438322735382003712)]} 0 4
[junit4:junit4] 2> 1091565 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11]} 0 7
[junit4:junit4] 2> 1091567 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12 (1438322735389343744)]} 0 0
[junit4:junit4] 2> 1091572 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!12 (1438322735391440896)]} 0 0
[junit4:junit4] 2> 1091573 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12 (1438322735391440896)]} 0 4
[junit4:junit4] 2> 1091576 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13 (1438322735398780928)]} 0 0
[junit4:junit4] 2> 1091583 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!13 (1438322735402975232)]} 0 0
[junit4:junit4] 2> 1091584 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!13 (1438322735402975232)]} 0 4
[junit4:junit4] 2> 1091585 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13]} 0 6
[junit4:junit4] 2> 1091587 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14 (1438322735410315264)]} 0 0
[junit4:junit4] 2> 1091593 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!14 (1438322735413460992)]} 0 0
[junit4:junit4] 2> 1091594 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14 (1438322735413460992)]} 0 4
[junit4:junit4] 2> 1091597 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438322735420801024)]} 0 0
[junit4:junit4] 2> 1091602 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!15 (1438322735423946752)]} 0 0
[junit4:junit4] 2> 1091603 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438322735423946752)]} 0 3
[junit4:junit4] 2> 1091606 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16 (1438322735430238208)]} 0 0
[junit4:junit4] 2> 1091613 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!16 (1438322735435481088)]} 0 0
[junit4:junit4] 2> 1091614 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!16 (1438322735435481088)]} 0 4
[junit4:junit4] 2> 1091615 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16]} 0 6
[junit4:junit4] 2> 1091617 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17 (1438322735441772544)]} 0 0
[junit4:junit4] 2> 1091625 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!17 (1438322735447015424)]} 0 0
[junit4:junit4] 2> 1091626 T2675 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!17 (1438322735447015424)]} 0 4
[junit4:junit4] 2> 1091627 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17]} 0 7
[junit4:junit4] 2> 1091629 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438322735454355456)]} 0 0
[junit4:junit4] 2> 1091635 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!18 (1438322735457501184)]} 0 0
[junit4:junit4] 2> 1091636 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438322735457501184)]} 0 4
[junit4:junit4] 2> 1091639 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19 (1438322735464841216)]} 0 0
[junit4:junit4] 2> 1091647 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!19 (1438322735470084096)]} 0 0
[junit4:junit4] 2> 1091648 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!19 (1438322735470084096)]} 0 4
[junit4:junit4] 2> 1091649 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19]} 0 7
[junit4:junit4] 2> 1091651 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20 (1438322735477424128)]} 0 0
[junit4:junit4] 2> 1091659 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!20 (1438322735482667008)]} 0 0
[junit4:junit4] 2> 1091660 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!20 (1438322735482667008)]} 0 4
[junit4:junit4] 2> 1091661 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20]} 0 7
[junit4:junit4] 2> 1091663 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21 (1438322735490007040)]} 0 0
[junit4:junit4] 2> 1091671 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!21 (1438322735495249920)]} 0 0
[junit4:junit4] 2> 1091672 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!21 (1438322735495249920)]} 0 4
[junit4:junit4] 2> 1091673 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21]} 0 7
[junit4:junit4] 2> 1091675 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438322735502589952)]} 0 0
[junit4:junit4] 2> 1091683 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!22 (1438322735507832832)]} 0 0
[junit4:junit4] 2> 1091684 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!22 (1438322735507832832)]} 0 4
[junit4:junit4] 2> 1091685 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22]} 0 7
[junit4:junit4] 2> 1091687 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23 (1438322735515172864)]} 0 0
[junit4:junit4] 2> 1091695 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!23 (1438322735520415744)]} 0 0
[junit4:junit4] 2> 1091696 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!23 (1438322735520415744)]} 0 4
[junit4:junit4] 2> 1091697 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23]} 0 7
[junit4:junit4] 2> 1091699 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438322735527755776)]} 0 0
[junit4:junit4] 2> 1091704 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!24 (1438322735530901504)]} 0 0
[junit4:junit4] 2> 1091705 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438322735530901504)]} 0 3
[junit4:junit4] 2> 1091708 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438322735537192960)]} 0 0
[junit4:junit4] 2> 1091714 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!25 (1438322735540338688)]} 0 0
[junit4:junit4] 2> 1091715 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438322735540338688)]} 0 4
[junit4:junit4] 2> 1091718 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26 (1438322735547678720)]} 0 0
[junit4:junit4] 2> 1091726 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!26 (1438322735552921600)]} 0 0
[junit4:junit4] 2> 1091727 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!26 (1438322735552921600)]} 0 4
[junit4:junit4] 2> 1091728 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26]} 0 7
[junit4:junit4] 2> 1091730 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27 (1438322735560261632)]} 0 0
[junit4:junit4] 2> 1091736 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!27 (1438322735563407360)]} 0 0
[junit4:junit4] 2> 1091737 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27 (1438322735563407360)]} 0 4
[junit4:junit4] 2> 1091740 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438322735570747392)]} 0 0
[junit4:junit4] 2> 1091746 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!28 (1438322735573893120)]} 0 0
[junit4:junit4] 2> 1091747 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438322735573893120)]} 0 4
[junit4:junit4] 2> 1091750 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438322735581233152)]} 0 0
[junit4:junit4] 2> 1091757 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!29 (1438322735586476032)]} 0 0
[junit4:junit4] 2> 1091758 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!29 (1438322735586476032)]} 0 3
[junit4:junit4] 2> 1091759 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29]} 0 6
[junit4:junit4] 2> 1091761 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30 (1438322735592767488)]} 0 0
[junit4:junit4] 2> 1091768 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!30 (1438322735598010368)]} 0 0
[junit4:junit4] 2> 1091769 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!30 (1438322735598010368)]} 0 3
[junit4:junit4] 2> 1091770 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30]} 0 6
[junit4:junit4] 2> 1091772 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31 (1438322735604301824)]} 0 0
[junit4:junit4] 2> 1091780 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!31 (1438322735609544704)]} 0 0
[junit4:junit4] 2> 1091781 T2675 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!31 (1438322735609544704)]} 0 4
[junit4:junit4] 2> 1091782 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31]} 0 7
[junit4:junit4] 2> 1091784 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32 (1438322735616884736)]} 0 0
[junit4:junit4] 2> 1091792 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!32 (1438322735622127616)]} 0 0
[junit4:junit4] 2> 1091793 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!32 (1438322735622127616)]} 0 4
[junit4:junit4] 2> 1091794 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32]} 0 7
[junit4:junit4] 2> 1091796 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33 (1438322735629467648)]} 0 0
[junit4:junit4] 2> 1091804 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!33 (1438322735634710528)]} 0 0
[junit4:junit4] 2> 1091805 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!33 (1438322735634710528)]} 0 4
[junit4:junit4] 2> 1091806 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33]} 0 7
[junit4:junit4] 2> 1091808 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34 (1438322735642050560)]} 0 0
[junit4:junit4] 2> 1091815 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!34 (1438322735647293440)]} 0 0
[junit4:junit4] 2> 1091816 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!34 (1438322735647293440)]} 0 3
[junit4:junit4] 2> 1091817 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34]} 0 6
[junit4:junit4] 2> 1091819 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438322735653584896)]} 0 0
[junit4:junit4] 2> 1091826 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!35 (1438322735658827776)]} 0 0
[junit4:junit4] 2> 1091827 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!35 (1438322735658827776)]} 0 3
[junit4:junit4] 2> 1091828 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35]} 0 6
[junit4:junit4] 2> 1091830 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36 (1438322735665119232)]} 0 0
[junit4:junit4] 2> 1091838 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!36 (1438322735670362112)]} 0 0
[junit4:junit4] 2> 1091839 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!36 (1438322735670362112)]} 0 4
[junit4:junit4] 2> 1091840 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36]} 0 7
[junit4:junit4] 2> 1091842 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37 (1438322735677702144)]} 0 0
[junit4:junit4] 2> 1091848 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!37 (1438322735680847872)]} 0 0
[junit4:junit4] 2> 1091849 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37 (1438322735680847872)]} 0 4
[junit4:junit4] 2> 1091852 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438322735688187904)]} 0 0
[junit4:junit4] 2> 1091857 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!38 (1438322735691333632)]} 0 0
[junit4:junit4] 2> 1091858 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438322735691333632)]} 0 3
[junit4:junit4] 2> 1091861 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39 (1438322735697625088)]} 0 0
[junit4:junit4] 2> 1091868 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!39 (1438322735702867968)]} 0 0
[junit4:junit4] 2> 1091869 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!39 (1438322735702867968)]} 0 3
[junit4:junit4] 2> 1091870 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39]} 0 6
[junit4:junit4] 2> 1091872 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40 (1438322735709159424)]} 0 0
[junit4:junit4] 2> 1091880 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!40 (1438322735714402304)]} 0 0
[junit4:junit4] 2> 1091881 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!40 (1438322735714402304)]} 0 4
[junit4:junit4] 2> 1091882 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40]} 0 7
[junit4:junit4] 2> 1091884 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41 (1438322735721742336)]} 0 0
[junit4:junit4] 2> 1091893 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!41 (1438322735726985216)]} 0 1
[junit4:junit4] 2> 1091893 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!41 (1438322735726985216)]} 0 4
[junit4:junit4] 2> 1091894 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41]} 0 7
[junit4:junit4] 2> 1091897 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42 (1438322735735373824)]} 0 0
[junit4:junit4] 2> 1091905 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!42 (1438322735740616704)]} 0 0
[junit4:junit4] 2> 1091906 T2734 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!42 (1438322735740616704)]} 0 4
[junit4:junit4] 2> 1091907 T2711 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42]} 0 7
[junit4:junit4] 2> 1091911 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43 (1438322735749005312)]} 0 1
[junit4:junit4] 2> 1091919 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!43 (1438322735755296768)]} 0 0
[junit4:junit4] 2> 1091920 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!43 (1438322735755296768)]} 0 4
[junit4:junit4] 2> 1091921 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43]} 0 7
[junit4:junit4] 2> 1091923 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44 (1438322735762636800)]} 0 0
[junit4:junit4] 2> 1091930 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!44 (1438322735767879680)]} 0 0
[junit4:junit4] 2> 1091931 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[s!44 (1438322735767879680)]} 0 3
[junit4:junit4] 2> 1091932 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44]} 0 6
[junit4:junit4] 2> 1091934 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45 (1438322735774171136)]} 0 0
[junit4:junit4] 2> 1091942 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!45 (1438322735779414016)]} 0 0
[junit4:junit4] 2> 1091943 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!45 (1438322735779414016)]} 0 4
[junit4:junit4] 2> 1091944 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45]} 0 7
[junit4:junit4] 2> 1091946 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46 (1438322735786754048)]} 0 0
[junit4:junit4] 2> 1091953 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!46 (1438322735791996928)]} 0 0
[junit4:junit4] 2> 1091954 T2681 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!46 (1438322735791996928)]} 0 3
[junit4:junit4] 2> 1091955 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46]} 0 6
[junit4:junit4] 2> 1091957 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47 (1438322735798288384)]} 0 0
[junit4:junit4] 2> 1091964 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!47 (1438322735803531264)]} 0 0
[junit4:junit4] 2> 1091965 T2661 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!47 (1438322735803531264)]} 0 3
[junit4:junit4] 2> 1091966 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47]} 0 6
[junit4:junit4] 2> 1091968 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48 (1438322735809822720)]} 0 0
[junit4:junit4] 2> 1091976 T2691 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!48 (1438322735815065600)]} 0 0
[junit4:junit4] 2> 1091977 T2659 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:16528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!48 (1438322735815065600)]} 0 4
[junit4:junit4] 2> 1091978 T2693 C480 P16528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48]} 0 7
[junit4:junit4] 2> 1091980 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49 (1438322735822405632)]} 0 0
[junit4:junit4] 2> 1091988 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!49 (1438322735827648512)]} 0 0
[junit4:junit4] 2> 1091989 T2679 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50770/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!49 (1438322735827648512)]} 0 4
[junit4:junit4] 2> 1091990 T2663 C481 P50770 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49]} 0 7
[junit4:junit4] 2> 1091992 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438322735834988544)]} 0 0
[junit4:junit4] 2> 1091998 T2710 C482 P60420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56319/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!50 (1438322735838134272)]} 0 0
[junit4:junit4] 2> 1091999 T2677 C483 P56319 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438322735838134272)]} 0 4
[junit4:junit4] 2> 1092002 T2639 C484 P41800 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51 (1438322735845474304)]} 0 0
[junit4:junit4
[...truncated too long message...]
://127.0.0.1:46949/d/pl"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:46907_d%2Fpl_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:46907_d%2Fpl",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46907/d/pl",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:46907_d%2Fpl (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:46949_d%2Fpl (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:46917_d%2Fpl (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:46930_d%2Fpl (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89895186462867466-127.0.0.1:46930_d%2Fpl_collection1-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89895186462867468-127.0.0.1:46949_d%2Fpl_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89895186462867470-127.0.0.1:46917_d%2Fpl_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:46930_d%2Fpl",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46930/d/pl"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89895186462867459-127.0.0.1:46907_d%2Fpl_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:46907_d%2Fpl",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46907/d/pl"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89895186462867466-127.0.0.1:46930_d%2Fpl-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89895186462867459-127.0.0.1:46907_d%2Fpl-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89895186462867468-127.0.0.1:46949_d%2Fpl-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89895186462867470-127.0.0.1:46917_d%2Fpl-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89895186462867459-127.0.0.1:46907_d%2Fpl-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=E26826E5A65C9485 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=America/Swift_Current -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 811s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:46917/d/pl/collection1lastClient and got 263 from http://127.0.0.1:46930/d/pl/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([E26826E5A65C9485:638EA8FDD103F4B9]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2139176 T2758 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 811102 T2757 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 2139223 T2852 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2139223 T2852 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2139223 T2852 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=lt, timezone=America/Swift_Current
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=1,free=43863520,total=85983232
[junit4:junit4] 2> NOTE: All tests run in this JVM: [WordBreakSolrSpellCheckerTest, JSONWriterTest, TestReload, SystemInfoHandlerTest, LeaderElectionIntegrationTest, NoCacheHeaderTest, DirectSolrSpellCheckerTest, SoftAutoCommitTest, TestSolr4Spatial, TestFiltering, TestCoreDiscovery, TestLMJelinekMercerSimilarityFactory, BadIndexSchemaTest, TestSearchPerf, MoreLikeThisHandlerTest, DOMUtilTest, StandardRequestHandlerTest, FileBasedSpellCheckerTest, TestBM25SimilarityFactory, SpellingQueryConverterTest, DocumentAnalysisRequestHandlerTest, ClusterStateTest, ZkSolrClientTest, TestStressLucene, TestQueryUtils, DefaultValueUpdateProcessorTest, TestCoreContainer, UniqFieldsUpdateProcessorFactoryTest, CacheHeaderTest, ChaosMonkeySafeLeaderTest, TestSolrDeletionPolicy1, SuggesterFSTTest, UnloadDistributedZkTest, TestSolrXml, FullSolrCloudDistribCmdsTest, PeerSyncTest, TestRecovery, TestSurroundQueryParser, CopyFieldTest, TestSchemaNameResource, TestSolrIndexConfig, TestPhraseSuggestions, ScriptEngineTest, SimpleFacetsTest, TestSolrQueryParserResource, TestAddFieldRealTimeGet, DistributedSpellCheckComponentTest, CachingDirectoryFactoryTest, SolrIndexConfigTest, TestReplicationHandler, ZkCLITest, SliceStateUpdateTest, TestPerFieldSimilarity, UpdateParamsTest, SuggesterTest, TestIBSimilarityFactory, TestArbitraryIndexDir, DateMathParserTest, TestMergePolicyConfig, StatsComponentTest, CurrencyFieldXmlFileTest, TestOmitPositions, SuggesterTSTTest, ShowFileRequestHandlerTest, PreAnalyzedUpdateProcessorTest, QueryParsingTest, IndexSchemaRuntimeFieldTest, TestLFUCache, TestCloudManagedSchema, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, SpellPossibilityIteratorTest, SolrRequestParserTest, EchoParamsTest, ChaosMonkeyShardSplitTest, ShardSplitTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 811.39s, 1 test, 1 failure <<< FAILURES!
[...truncated 411 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1246: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:890: There were test failures: 298 suites, 1255 tests, 1 error, 2 failures, 13 ignored (7 assumptions)
Total time: 58 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1343 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1343/
2 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:38692/collection1lastClient and got 234 from http://127.0.0.1:31872/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:38692/collection1lastClient and got 234 from http://127.0.0.1:31872/collection1
at __randomizedtesting.SeedInfo.seed([BA912B0BBB7987CE:3B77A513CC26E7F2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:29823 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:29823 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([BA912B0BBB7987CE:3B77A513CC26E7F2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9506 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4:junit4] 2> 345903 T913 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 345910 T913 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1371658976718
[junit4:junit4] 2> 345922 T913 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 345930 T914 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 346132 T913 oasc.ZkTestServer.run start zk server on port:52195
[junit4:junit4] 2> 346418 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 346470 T915 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4] 2> 348172 T918 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71cecb16 name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 348172 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 348173 T913 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 348282 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 348285 T922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@413dbfc7 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 348285 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 348286 T913 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 348292 T913 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 348302 T913 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 348306 T913 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 348311 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 348312 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 348326 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 348327 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 348438 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 348439 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 348450 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 348451 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 348456 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 348457 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 348462 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 348462 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 348467 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 348468 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 348473 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 348474 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 348479 T913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 348480 T913 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 348755 T913 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 348761 T913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:43958
[junit4:junit4] 2> 348761 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 348762 T913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 348762 T913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293
[junit4:junit4] 2> 348763 T913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/solr.xml
[junit4:junit4] 2> 348763 T913 oasc.CoreContainer.<init> New CoreContainer 788730623
[junit4:junit4] 2> 348764 T913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/'
[junit4:junit4] 2> 348764 T913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/'
[junit4:junit4] 2> 348911 T913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 348912 T913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 348912 T913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 348913 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 348913 T913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 348914 T913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 348914 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 348915 T913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 348915 T913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 348916 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 348934 T913 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 348935 T913 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52195/solr
[junit4:junit4] 2> 348966 T913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 348967 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 348971 T933 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ab1a317 name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 348971 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 348980 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 348999 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 349002 T935 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59734cf1 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 349002 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 349011 T913 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 349029 T913 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 349046 T913 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 349049 T913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43958_
[junit4:junit4] 2> 349058 T913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43958_
[junit4:junit4] 2> 349063 T913 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 349097 T913 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 349101 T913 oasc.Overseer.start Overseer (id=89893042724798467-127.0.0.1:43958_-n_0000000000) starting
[junit4:junit4] 2> 349130 T913 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 349137 T937 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 349137 T913 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 349141 T913 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 349149 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 349172 T936 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 349180 T938 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 349181 T938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 350678 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 350680 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43958_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43958"}
[junit4:junit4] 2> 350680 T936 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 350709 T936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 350737 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 351186 T938 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/collection1
[junit4:junit4] 2> 351186 T938 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 351187 T938 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 351187 T938 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 351190 T938 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/collection1/'
[junit4:junit4] 2> 351191 T938 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 351192 T938 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/collection1/lib/README' to classloader
[junit4:junit4] 2> 351280 T938 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 351343 T938 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 351445 T938 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 351451 T938 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 352014 T938 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 352017 T938 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 352019 T938 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 352023 T938 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 352057 T938 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 352058 T938 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371658979293/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data/
[junit4:junit4] 2> 352058 T938 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18d35e27
[junit4:junit4] 2> 352058 T938 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 352059 T938 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data
[junit4:junit4] 2> 352059 T938 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data/index/
[junit4:junit4] 2> 352059 T938 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 352059 T938 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data/index
[junit4:junit4] 2> 352063 T938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@240ab5e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21f0785e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 352063 T938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 352065 T938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 352065 T938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 352066 T938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 352066 T938 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 352067 T938 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 352067 T938 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 352067 T938 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 352067 T938 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 352068 T938 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 352078 T938 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 352083 T938 oass.SolrIndexSearcher.<init> Opening Searcher@795f015d main
[junit4:junit4] 2> 352084 T938 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/control/data/tlog
[junit4:junit4] 2> 352084 T938 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 352085 T938 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 352087 T939 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@795f015d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 352088 T938 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 352088 T938 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43958 collection:control_collection shard:shard1
[junit4:junit4] 2> 352097 T938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 352110 T938 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 352130 T938 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 352141 T938 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 352141 T938 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 352142 T938 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43958/collection1/
[junit4:junit4] 2> 352143 T938 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 352143 T938 oasc.SyncStrategy.syncToMe http://127.0.0.1:43958/collection1/ has no replicas
[junit4:junit4] 2> 352143 T938 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43958/collection1/
[junit4:junit4] 2> 352143 T938 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 352244 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 352262 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 352306 T938 oasc.ZkController.register We are http://127.0.0.1:43958/collection1/ and leader is http://127.0.0.1:43958/collection1/
[junit4:junit4] 2> 352306 T938 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43958
[junit4:junit4] 2> 352306 T938 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 352307 T938 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 352307 T938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 352309 T938 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 352310 T913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 352311 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 352311 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 352321 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 352326 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 352328 T942 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@80c3068 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 352328 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 352330 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 352337 T913 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 352604 T913 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 352607 T913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:29823
[junit4:junit4] 2> 352608 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 352609 T913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 352609 T913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144
[junit4:junit4] 2> 352610 T913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/solr.xml
[junit4:junit4] 2> 352610 T913 oasc.CoreContainer.<init> New CoreContainer 1498732458
[junit4:junit4] 2> 352611 T913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/'
[junit4:junit4] 2> 352611 T913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/'
[junit4:junit4] 2> 352743 T913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 352744 T913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 352744 T913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 352745 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 352745 T913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 352746 T913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 352746 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 352747 T913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 352747 T913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 352748 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 352766 T913 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 352767 T913 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52195/solr
[junit4:junit4] 2> 352767 T913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 352769 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 352772 T953 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60cf85c0 name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 352772 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 352789 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 352799 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 352841 T955 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ef82514 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 352841 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 352914 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 353777 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 353778 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43958_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43958"}
[junit4:junit4] 2> 353817 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353817 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353817 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353919 T913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:29823_
[junit4:junit4] 2> 353921 T913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:29823_
[junit4:junit4] 2> 353925 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353925 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353925 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353927 T935 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 353928 T942 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 353927 T955 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 353940 T956 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 353941 T956 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 355324 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 355326 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29823_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29823"}
[junit4:junit4] 2> 355326 T936 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 355326 T936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 355348 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 355348 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 355348 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 355945 T956 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/collection1
[junit4:junit4] 2> 355945 T956 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 355946 T956 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 355946 T956 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 355949 T956 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/collection1/'
[junit4:junit4] 2> 355950 T956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 355951 T956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/collection1/lib/README' to classloader
[junit4:junit4] 2> 356005 T956 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 356067 T956 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 356169 T956 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 356174 T956 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 356756 T956 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 356760 T956 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 356762 T956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 356768 T956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 356806 T956 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 356806 T956 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371658983144/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1/
[junit4:junit4] 2> 356806 T956 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18d35e27
[junit4:junit4] 2> 356807 T956 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 356808 T956 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1
[junit4:junit4] 2> 356808 T956 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1/index/
[junit4:junit4] 2> 356808 T956 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 356809 T956 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1/index
[junit4:junit4] 2> 356813 T956 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_1,generation=1}
[junit4:junit4] 2> 356814 T956 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 356817 T956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 356817 T956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 356818 T956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 356819 T956 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 356819 T956 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 356820 T956 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 356820 T956 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 356821 T956 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 356821 T956 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 356838 T956 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 356845 T956 oass.SolrIndexSearcher.<init> Opening Searcher@7e20ee12 main
[junit4:junit4] 2> 356846 T956 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty1/tlog
[junit4:junit4] 2> 356847 T956 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 356847 T956 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 356852 T957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e20ee12 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 356854 T956 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 356854 T956 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:29823 collection:collection1 shard:shard1
[junit4:junit4] 2> 356855 T956 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 356876 T956 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 356880 T956 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 356880 T956 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 356880 T956 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:29823/collection1/
[junit4:junit4] 2> 356881 T956 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 356881 T956 oasc.SyncStrategy.syncToMe http://127.0.0.1:29823/collection1/ has no replicas
[junit4:junit4] 2> 356881 T956 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:29823/collection1/
[junit4:junit4] 2> 356881 T956 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 358358 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 358383 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 358383 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 358383 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 358421 T956 oasc.ZkController.register We are http://127.0.0.1:29823/collection1/ and leader is http://127.0.0.1:29823/collection1/
[junit4:junit4] 2> 358421 T956 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:29823
[junit4:junit4] 2> 358421 T956 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 358421 T956 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 358422 T956 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 358424 T956 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 358426 T913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 358426 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 358427 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 358707 T913 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 358710 T913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11776
[junit4:junit4] 2> 358711 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 358711 T913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 358712 T913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253
[junit4:junit4] 2> 358712 T913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/solr.xml
[junit4:junit4] 2> 358713 T913 oasc.CoreContainer.<init> New CoreContainer 1833864197
[junit4:junit4] 2> 358714 T913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/'
[junit4:junit4] 2> 358714 T913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/'
[junit4:junit4] 2> 358843 T913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 358844 T913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 358845 T913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 358845 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 358846 T913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 358846 T913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 358847 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 358847 T913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 358848 T913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 358848 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 358867 T913 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 358868 T913 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52195/solr
[junit4:junit4] 2> 358868 T913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 358869 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 358873 T969 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21bb6eb8 name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358874 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 358889 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 358902 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 358905 T971 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35f9b1fd name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358905 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 358917 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 359897 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 359898 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29823_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29823"}
[junit4:junit4] 2> 359914 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359914 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359914 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359914 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359922 T913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11776_
[junit4:junit4] 2> 359924 T913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11776_
[junit4:junit4] 2> 359928 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359928 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359928 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359928 T971 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 359929 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 359930 T955 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 359930 T942 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 359931 T935 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 359938 T972 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 359938 T972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 361421 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 361423 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11776_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11776"}
[junit4:junit4] 2> 361423 T936 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 361423 T936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 361439 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 361439 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 361439 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 361439 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 361942 T972 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/collection1
[junit4:junit4] 2> 361942 T972 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 361943 T972 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 361943 T972 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 361946 T972 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/collection1/'
[junit4:junit4] 2> 361947 T972 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 361948 T972 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/collection1/lib/README' to classloader
[junit4:junit4] 2> 362002 T972 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 362064 T972 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 362166 T972 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 362172 T972 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 362753 T972 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 362758 T972 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 362760 T972 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 362766 T972 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 362806 T972 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 362806 T972 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371658989253/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2/
[junit4:junit4] 2> 362807 T972 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18d35e27
[junit4:junit4] 2> 362807 T972 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 362809 T972 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2
[junit4:junit4] 2> 362809 T972 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2/index/
[junit4:junit4] 2> 362809 T972 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 362810 T972 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2/index
[junit4:junit4] 2> 362813 T972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_1,generation=1}
[junit4:junit4] 2> 362814 T972 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 362817 T972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 362817 T972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 362818 T972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 362819 T972 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 362820 T972 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 362820 T972 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 362820 T972 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 362821 T972 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 362821 T972 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 362838 T972 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 362846 T972 oass.SolrIndexSearcher.<init> Opening Searcher@55834b main
[junit4:junit4] 2> 362846 T972 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty2/tlog
[junit4:junit4] 2> 362847 T972 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 362848 T972 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 362852 T973 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55834b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 362855 T972 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 362855 T972 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11776 collection:collection1 shard:shard2
[junit4:junit4] 2> 362856 T972 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 362892 T972 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 362916 T972 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 362917 T972 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 362917 T972 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11776/collection1/
[junit4:junit4] 2> 362917 T972 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 362917 T972 oasc.SyncStrategy.syncToMe http://127.0.0.1:11776/collection1/ has no replicas
[junit4:junit4] 2> 362918 T972 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11776/collection1/
[junit4:junit4] 2> 362918 T972 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 362958 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 363110 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 363111 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 363111 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 363111 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 363118 T972 oasc.ZkController.register We are http://127.0.0.1:11776/collection1/ and leader is http://127.0.0.1:11776/collection1/
[junit4:junit4] 2> 363118 T972 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11776
[junit4:junit4] 2> 363118 T972 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 363118 T972 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 363119 T972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 363121 T972 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 363122 T913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 363123 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 363124 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 363403 T913 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 363407 T913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:53094
[junit4:junit4] 2> 363408 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 363408 T913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 363409 T913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949
[junit4:junit4] 2> 363409 T913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/solr.xml
[junit4:junit4] 2> 363410 T913 oasc.CoreContainer.<init> New CoreContainer 2092537837
[junit4:junit4] 2> 363410 T913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/'
[junit4:junit4] 2> 363411 T913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/'
[junit4:junit4] 2> 363556 T913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 363557 T913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 363557 T913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 363558 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 363558 T913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 363559 T913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 363559 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 363560 T913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 363560 T913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 363561 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 363580 T913 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 363581 T913 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52195/solr
[junit4:junit4] 2> 363582 T913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 363583 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 363588 T985 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27d918cc name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 363589 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 363603 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 363617 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 363619 T987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6013a76 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 363620 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 363633 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 364620 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 364621 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11776_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11776"}
[junit4:junit4] 2> 364636 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364636 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364636 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364636 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364636 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364639 T913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53094_
[junit4:junit4] 2> 364646 T913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53094_
[junit4:junit4] 2> 364650 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364650 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364650 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364650 T971 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 364650 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364651 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 364653 T942 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 364653 T935 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 364653 T987 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 364653 T955 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 364661 T988 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 364661 T988 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 366150 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 366151 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53094_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53094"}
[junit4:junit4] 2> 366151 T936 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 366152 T936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 366166 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366166 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366166 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366166 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366166 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366665 T988 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/collection1
[junit4:junit4] 2> 366665 T988 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 366666 T988 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 366666 T988 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 366669 T988 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/collection1/'
[junit4:junit4] 2> 366670 T988 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/collection1/lib/README' to classloader
[junit4:junit4] 2> 366671 T988 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 366724 T988 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 366786 T988 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 366888 T988 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 366894 T988 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 367466 T988 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 367469 T988 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 367470 T988 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 367475 T988 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 367509 T988 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 367509 T988 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371658993949/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/
[junit4:junit4] 2> 367509 T988 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18d35e27
[junit4:junit4] 2> 367510 T988 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 367510 T988 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3
[junit4:junit4] 2> 367510 T988 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index/
[junit4:junit4] 2> 367511 T988 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 367511 T988 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index
[junit4:junit4] 2> 367514 T988 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55d7d510 lockFactory=org.apache.lucene.store.NativeFSLockFactory@566768da),segFN=segments_1,generation=1}
[junit4:junit4] 2> 367514 T988 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 367516 T988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 367516 T988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 367517 T988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 367517 T988 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 367518 T988 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 367518 T988 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 367518 T988 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 367519 T988 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 367519 T988 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 367529 T988 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 367534 T988 oass.SolrIndexSearcher.<init> Opening Searcher@6b30632e main
[junit4:junit4] 2> 367535 T988 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/tlog
[junit4:junit4] 2> 367535 T988 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 367536 T988 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 367539 T989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b30632e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 367541 T988 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 367541 T988 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53094 collection:collection1 shard:shard1
[junit4:junit4] 2> 367546 T988 oasc.ZkController.register We are http://127.0.0.1:53094/collection1/ and leader is http://127.0.0.1:29823/collection1/
[junit4:junit4] 2> 367546 T988 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53094
[junit4:junit4] 2> 367546 T988 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 367546 T988 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1725 name=collection1 org.apache.solr.core.SolrCore@9ad082c url=http://127.0.0.1:53094/collection1 node=127.0.0.1:53094_ C1725_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:53094_, base_url=http://127.0.0.1:53094}
[junit4:junit4] 2> 367552 T990 C1725 P53094 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 367552 T988 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 367552 T990 C1725 P53094 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 367553 T990 C1725 P53094 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 367553 T990 C1725 P53094 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 367554 T913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 367554 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 367554 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 367555 T990 C1725 P53094 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 367583 T945 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 367673 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 367674 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53094_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53094"}
[junit4:junit4] 2> 367690 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367690 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367690 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367690 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367690 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367747 T913 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 367749 T913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:20360
[junit4:junit4] 2> 367750 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 367750 T913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 367750 T913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377
[junit4:junit4] 2> 367751 T913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/solr.xml
[junit4:junit4] 2> 367751 T913 oasc.CoreContainer.<init> New CoreContainer 1223718706
[junit4:junit4] 2> 367751 T913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/'
[junit4:junit4] 2> 367752 T913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/'
[junit4:junit4] 2> 367841 T913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 367841 T913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 367842 T913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 367842 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 367842 T913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 367843 T913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 367843 T913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 367843 T913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 367844 T913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 367844 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 367856 T913 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 367857 T913 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52195/solr
[junit4:junit4] 2> 367857 T913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 367858 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 367874 T1002 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bf80519 name:ZooKeeperConnection Watcher:127.0.0.1:52195 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 367875 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 367883 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 367897 T913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 367902 T1004 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9620807 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 367902 T913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 367945 T913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 368585 T945 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 368585 T945 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:53094_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 368951 T913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:20360_
[junit4:junit4] 2> 368954 T913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:20360_
[junit4:junit4] 2> 368958 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368958 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368958 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368959 T971 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368959 T1004 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368958 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368959 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368961 T987 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368961 T955 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368961 T942 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368973 T935 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368979 T1005 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 368979 T1005 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 369196 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 369197 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20360_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20360"}
[junit4:junit4] 2> 369198 T936 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 369198 T936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 369211 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369211 T1004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369211 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369211 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369211 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369211 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369982 T1005 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/collection1
[junit4:junit4] 2> 369982 T1005 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 369983 T1005 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 369983 T1005 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 369986 T1005 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/collection1/'
[junit4:junit4] 2> 369987 T1005 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 369988 T1005 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/collection1/lib/README' to classloader
[junit4:junit4] 2> 370040 T1005 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 370102 T1005 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 370203 T1005 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 370208 T1005 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1726 name=collection1 org.apache.solr.core.SolrCore@9ad082c url=http://127.0.0.1:53094/collection1 node=127.0.0.1:53094_ C1726_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:53094_, base_url=http://127.0.0.1:53094}
[junit4:junit4] 2> 370587 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:29823/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 370601 T990 C1726 P53094 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 370624 T990 C1726 P53094 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:53094 START replicas=[http://127.0.0.1:29823/collection1/] nUpdates=100
[junit4:junit4] 2> 370624 T990 C1726 P53094 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 370625 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 370626 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 370626 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 370626 T990 C1726 P53094 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 370626 T990 C1726 P53094 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:29823/collection1/. core=collection1
[junit4:junit4] 2> 370627 T990 C1726 P53094 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1727 name=collection1 org.apache.solr.core.SolrCore@52e8a6cb url=http://127.0.0.1:29823/collection1 node=127.0.0.1:29823_ C1727_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:29823_, base_url=http://127.0.0.1:29823, leader=true}
[junit4:junit4] 2> 370635 T946 C1727 P29823 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 370656 T947 C1727 P29823 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 370659 T947 C1727 P29823 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_1,generation=1}
[junit4:junit4] 2> 370660 T947 C1727 P29823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 370661 T947 C1727 P29823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_2,generation=2}
[junit4:junit4] 2> 370661 T947 C1727 P29823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 370662 T947 C1727 P29823 oass.SolrIndexSearcher.<init> Opening Searcher@10dce187 realtime
[junit4:junit4] 2> 370662 T947 C1727 P29823 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 370663 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 370753 T990 C1726 P53094 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 370754 T990 C1726 P53094 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 370779 T990 C1726 P53094 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 370782 T948 C1727 P29823 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 370783 T948 C1727 P29823 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 370784 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 370785 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 370785 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 370787 T948 C1727 P29823 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 370788 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 370790 T990 C1726 P53094 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index.20130619135321596
[junit4:junit4] 2> 370790 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f73de5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44653401) fullCopy=false
[junit4:junit4] 2> 370807 T948 C1727 P29823 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=7
[junit4:junit4] 2> 370814 T990 C1726 P53094 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 370815 T990 C1726 P53094 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 370816 T990 C1726 P53094 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 370817 T1005 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 370821 T1005 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 370823 T1005 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 370825 T990 C1726 P53094 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55d7d510 lockFactory=org.apache.lucene.store.NativeFSLockFactory@566768da),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55d7d510 lockFactory=org.apache.lucene.store.NativeFSLockFactory@566768da),segFN=segments_2,generation=2}
[junit4:junit4] 2> 370826 T990 C1726 P53094 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 370826 T990 C1726 P53094 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 370827 T990 C1726 P53094 oass.SolrIndexSearcher.<init> Opening Searcher@10c6b580 main
[junit4:junit4] 2> 370828 T989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10c6b580 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 370829 T990 C1726 P53094 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index.20130619135321596 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index.20130619135321596;done=true>>]
[junit4:junit4] 2> 370829 T990 C1726 P53094 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index.20130619135321596
[junit4:junit4] 2> 370829 T990 C1726 P53094 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty3/index.20130619135321596
[junit4:junit4] 2> 370830 T990 C1726 P53094 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 370830 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 370830 T990 C1726 P53094 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 370830 T990 C1726 P53094 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 370832 T990 C1726 P53094 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 370845 T1005 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 370881 T1005 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 370882 T1005 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371658998377/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/
[junit4:junit4] 2> 370882 T1005 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18d35e27
[junit4:junit4] 2> 370883 T1005 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 370883 T1005 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4
[junit4:junit4] 2> 370884 T1005 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index/
[junit4:junit4] 2> 370884 T1005 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 370884 T1005 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index
[junit4:junit4] 2> 370888 T1005 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c48d365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22df031c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 370889 T1005 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 370892 T1005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 370892 T1005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 370893 T1005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 370894 T1005 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 370894 T1005 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 370895 T1005 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 370902 T1005 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 370902 T1005 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 370903 T1005 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 370919 T1005 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 370926 T1005 oass.SolrIndexSearcher.<init> Opening Searcher@304f8125 main
[junit4:junit4] 2> 370927 T1005 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/tlog
[junit4:junit4] 2> 370928 T1005 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 370928 T1005 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 370933 T1008 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@304f8125 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 370935 T1005 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 370936 T1005 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:20360 collection:collection1 shard:shard2
[junit4:junit4] 2> 370940 T1005 oasc.ZkController.register We are http://127.0.0.1:20360/collection1/ and leader is http://127.0.0.1:11776/collection1/
[junit4:junit4] 2> 370941 T1005 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:20360
[junit4:junit4] 2> 370941 T1005 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 370941 T1005 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1728 name=collection1 org.apache.solr.core.SolrCore@58ef45f url=http://127.0.0.1:20360/collection1 node=127.0.0.1:20360_ C1728_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:20360_, base_url=http://127.0.0.1:20360}
[junit4:junit4] 2> 370942 T1009 C1728 P20360 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 370943 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 370943 T1009 C1728 P20360 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 370943 T1009 C1728 P20360 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 370943 T1005 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370945 T1009 C1728 P20360 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 370946 T913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 370946 T913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 370947 T913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 370969 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370970 T961 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 370971 T913 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 370972 T913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 370973 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 371976 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 372221 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 372222 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53094_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53094"}
[junit4:junit4] 2> 372228 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20360_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20360"}
[junit4:junit4] 2> 372241 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372241 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372241 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372241 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372241 T1004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372241 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 372972 T961 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 372972 T961 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:20360_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 372979 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 373981 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1728_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:20360_, base_url=http://127.0.0.1:20360}
[junit4:junit4] 2> 374975 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:11776/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 374975 T1009 C1728 P20360 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:20360 START replicas=[http://127.0.0.1:11776/collection1/] nUpdates=100
[junit4:junit4] 2> 374976 T1009 C1728 P20360 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 374976 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 374976 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 374976 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 374976 T1009 C1728 P20360 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 374977 T1009 C1728 P20360 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:11776/collection1/. core=collection1
[junit4:junit4] 2> 374977 T1009 C1728 P20360 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 374984 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1729 name=collection1 org.apache.solr.core.SolrCore@57d326f1 url=http://127.0.0.1:11776/collection1 node=127.0.0.1:11776_ C1729_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:11776_, base_url=http://127.0.0.1:11776, leader=true}
[junit4:junit4] 2> 374985 T962 C1729 P11776 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 374991 T963 C1729 P11776 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 374993 T963 C1729 P11776 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_1,generation=1}
[junit4:junit4] 2> 374994 T963 C1729 P11776 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 374995 T963 C1729 P11776 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_2,generation=2}
[junit4:junit4] 2> 374995 T963 C1729 P11776 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 374996 T963 C1729 P11776 oass.SolrIndexSearcher.<init> Opening Searcher@57f8b83a realtime
[junit4:junit4] 2> 374996 T963 C1729 P11776 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 374997 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 374998 T1009 C1728 P20360 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 374998 T1009 C1728 P20360 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 375000 T964 C1729 P11776 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 375001 T964 C1729 P11776 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 375001 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 375002 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 375002 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 375004 T964 C1729 P11776 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 375005 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 375006 T1009 C1728 P20360 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index.20130619135325812
[junit4:junit4] 2> 375006 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@14e2d725 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5810543c) fullCopy=false
[junit4:junit4] 2> 375009 T964 C1729 P11776 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 375010 T1009 C1728 P20360 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 375011 T1009 C1728 P20360 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 375011 T1009 C1728 P20360 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 375013 T1009 C1728 P20360 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c48d365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22df031c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c48d365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22df031c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 375013 T1009 C1728 P20360 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 375014 T1009 C1728 P20360 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 375014 T1009 C1728 P20360 oass.SolrIndexSearcher.<init> Opening Searcher@4445d7aa main
[junit4:junit4] 2> 375015 T1008 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4445d7aa main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 375016 T1009 C1728 P20360 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index.20130619135325812 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index.20130619135325812;done=true>>]
[junit4:junit4] 2> 375016 T1009 C1728 P20360 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index.20130619135325812
[junit4:junit4] 2> 375016 T1009 C1728 P20360 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371658976717/jetty4/index.20130619135325812
[junit4:junit4] 2> 375017 T1009 C1728 P20360 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 375017 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 375017 T1009 C1728 P20360 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 375017 T1009 C1728 P20360 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 375019 T1009 C1728 P20360 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 375259 T936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 375260 T936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20360_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20360"}
[junit4:junit4] 2> 375275 T971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375275 T955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375275 T942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375275 T1004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375275 T987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375275 T935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 375987 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 375989 T913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1730 name=collection1 org.apache.solr.core.SolrCore@68a81833 url=http://127.0.0.1:43958/collection1 node=127.0.0.1:43958_ C1730_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:43958_, base_url=http://127.0.0.1:43958, leader=true}
[junit4:junit4] 2> 375999 T925 C1730 P43958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 376002 T925 C1730 P43958 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@240ab5e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21f0785e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 376003 T925 C1730 P43958 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 376004 T925 C1730 P43958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@240ab5e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21f0785e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@240ab5e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21f0785e),segFN=segments_2,generation=2}
[junit4:junit4] 2> 376004 T925 C1730 P43958 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 376005 T925 C1730 P43958 oass.SolrIndexSearcher.<init> Opening Searcher@46005a60 main
[junit4:junit4] 2> 376005 T925 C1730 P43958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 376006 T939 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46005a60 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 376007 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 376022 T965 C1729 P11776 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:29823/collection1/, StdNode: http://127.0.0.1:53094/collection1/, StdNode: http://127.0.0.1:11776/collection1/, StdNode: http://127.0.0.1:20360/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 376032 T966 C1729 P11776 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 376032 T949 C1727 P29823 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 376033 T966 C1729 P11776 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@171320e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fe28cfe),segFN=segments_3,generation=3}
[junit4:junit4] 2> 376034 T949 C1727 P29823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8aed6b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@171fdeff),segFN=segments_3,generation=3}
[junit4:junit4] 2> 376034 T966 C1729 P11776 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 376035 T949 C1727 P29823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 376035 T966 C1729 P11776 oass.SolrIndexSearcher.<init> Opening Searcher@4fe1cf3 main
[junit4:junit4] 2> 376036 T949 C1727 P29823 oass.SolrIndexSearcher.<init> Opening Searcher@29926d86 main
[junit4:junit4] 2> 376036 T966 C1729 P11776 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 376037 T949 C1727 P29823 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 376037 T973 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4fe1cf3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 376037 T957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@29926d86 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 376038 T966 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 376038 T949 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> C1728_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:20360_, base_url=http://127.0.0.1:20360}
[junit4:junit4] 2> 376040 T994 C1728 P20360 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1726_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:53094_, base_url=http://127.0.0.1:53094}
[junit4:junit4] 2> 376040 T977 C1726 P53094 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 376041 T994 C1728 P20360 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c48d365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22df031c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c48d365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22df031c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 376041 T977 C1726 P53094 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55d7d510 lockFactory=org.apache.lucene.store.NativeFSLockFactory@566768da),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55d7d510 lockFactory=org.apache.lucene.store.NativeFSLockFactory@566768da),segFN=segments_3,generation=3}
[junit4:junit4] 2> 376041 T994 C1728 P20360 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 376042 T977 C1726 P53094 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 376043 T994 C1728 P20360 oass.SolrIndexSearcher.<init> Opening Searcher@75d882dc main
[junit4:junit4] 2> 376043 T977 C1726 P53094 oass.SolrIndexSearcher.<init> Opening Searcher@58628312 main
[junit4:junit4] 2> 376044 T994 C1728 P20360 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 376044 T977 C1726 P53094 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 376045 T989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58628312 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 376044 T1008 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75d882dc main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 376045 T977 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 376045 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 376046 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 376048 T913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 376051 T950 C1727 P29823 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 376053 T978 C1726 P53094 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 376055 T967 C1729 P11776 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 376057 T995 C1728 P20360 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 378069 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438288716889063424)} 0 8
[junit4:junit4] 2> 378086 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438288716904792064&update.from=http://127.0.0.1:29823/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438288716904792064)} 0 1
[junit4:junit4] 2> 378086 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438288716904792064&update.from=http://127.0.0.1:11776/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438288716904792064)} 0 1
[junit4:junit4] 2> 378087 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438288716904792064)} 0 5
[junit4:junit4] 2> 378088 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438288716904792064)} 0 7
[junit4:junit4] 2> 378088 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 17
[junit4:junit4] 2> 378092 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0 (1438288716914229248)]} 0 1
[junit4:junit4] 2> 378119 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!0 (1438288716939395072)]} 0 1
[junit4:junit4] 2> 378119 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!0 (1438288716939395072)]} 0 4
[junit4:junit4] 2> 378120 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0]} 0 25
[junit4:junit4] 2> 378123 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1 (1438288716947783680)]} 0 0
[junit4:junit4] 2> 378133 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!1 (1438288716954075136)]} 0 1
[junit4:junit4] 2> 378133 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!1 (1438288716954075136)]} 0 4
[junit4:junit4] 2> 378134 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1]} 0 8
[junit4:junit4] 2> 378137 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2 (1438288716962463744)]} 0 0
[junit4:junit4] 2> 378144 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!2 (1438288716966658048)]} 0 0
[junit4:junit4] 2> 378145 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!2 (1438288716966658048)]} 0 4
[junit4:junit4] 2> 378146 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2]} 0 6
[junit4:junit4] 2> 378148 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3 (1438288716973998080)]} 0 0
[junit4:junit4] 2> 378155 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!3 (1438288716979240960)]} 0 0
[junit4:junit4] 2> 378156 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!3 (1438288716979240960)]} 0 3
[junit4:junit4] 2> 378157 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3]} 0 6
[junit4:junit4] 2> 378159 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4 (1438288716985532416)]} 0 0
[junit4:junit4] 2> 378166 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!4 (1438288716990775296)]} 0 0
[junit4:junit4] 2> 378167 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!4 (1438288716990775296)]} 0 3
[junit4:junit4] 2> 378167 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4]} 0 5
[junit4:junit4] 2> 378170 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5 (1438288716997066752)]} 0 0
[junit4:junit4] 2> 378177 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!5 (1438288717001261056)]} 0 0
[junit4:junit4] 2> 378178 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!5 (1438288717001261056)]} 0 4
[junit4:junit4] 2> 378178 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5]} 0 5
[junit4:junit4] 2> 378181 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438288717008601088)]} 0 0
[junit4:junit4] 2> 378189 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!6 (1438288717013843968)]} 0 0
[junit4:junit4] 2> 378190 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!6 (1438288717013843968)]} 0 4
[junit4:junit4] 2> 378191 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6]} 0 7
[junit4:junit4] 2> 378193 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7 (1438288717021184000)]} 0 0
[junit4:junit4] 2> 378200 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!7 (1438288717026426880)]} 0 0
[junit4:junit4] 2> 378201 T949 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!7 (1438288717026426880)]} 0 3
[junit4:junit4] 2> 378201 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7]} 0 5
[junit4:junit4] 2> 378204 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8 (1438288717032718336)]} 0 0
[junit4:junit4] 2> 378211 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!8 (1438288717037961216)]} 0 0
[junit4:junit4] 2> 378212 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!8 (1438288717037961216)]} 0 3
[junit4:junit4] 2> 378212 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8]} 0 5
[junit4:junit4] 2> 378215 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9 (1438288717044252672)]} 0 0
[junit4:junit4] 2> 378222 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!9 (1438288717049495552)]} 0 0
[junit4:junit4] 2> 378223 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!9 (1438288717049495552)]} 0 3
[junit4:junit4] 2> 378223 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9]} 0 5
[junit4:junit4] 2> 378226 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10 (1438288717055787008)]} 0 0
[junit4:junit4] 2> 378233 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!10 (1438288717061029888)]} 0 0
[junit4:junit4] 2> 378234 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!10 (1438288717061029888)]} 0 3
[junit4:junit4] 2> 378235 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10]} 0 6
[junit4:junit4] 2> 378237 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11 (1438288717067321344)]} 0 0
[junit4:junit4] 2> 378244 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!11 (1438288717072564224)]} 0 0
[junit4:junit4] 2> 378245 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!11 (1438288717072564224)]} 0 3
[junit4:junit4] 2> 378245 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11]} 0 5
[junit4:junit4] 2> 378248 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12 (1438288717078855680)]} 0 0
[junit4:junit4] 2> 378255 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!12 (1438288717084098560)]} 0 0
[junit4:junit4] 2> 378256 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!12 (1438288717084098560)]} 0 3
[junit4:junit4] 2> 378256 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12]} 0 5
[junit4:junit4] 2> 378259 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13 (1438288717090390016)]} 0 0
[junit4:junit4] 2> 378266 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!13 (1438288717095632896)]} 0 0
[junit4:junit4] 2> 378267 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!13 (1438288717095632896)]} 0 3
[junit4:junit4] 2> 378267 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13]} 0 5
[junit4:junit4] 2> 378270 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14 (1438288717101924352)]} 0 0
[junit4:junit4] 2> 378277 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!14 (1438288717107167232)]} 0 0
[junit4:junit4] 2> 378278 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!14 (1438288717107167232)]} 0 3
[junit4:junit4] 2> 378279 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14]} 0 6
[junit4:junit4] 2> 378281 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438288717113458688)]} 0 0
[junit4:junit4] 2> 378288 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!15 (1438288717118701568)]} 0 0
[junit4:junit4] 2> 378289 T949 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!15 (1438288717118701568)]} 0 3
[junit4:junit4] 2> 378289 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15]} 0 5
[junit4:junit4] 2> 378292 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16 (1438288717124993024)]} 0 0
[junit4:junit4] 2> 378299 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!16 (1438288717130235904)]} 0 0
[junit4:junit4] 2> 378300 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!16 (1438288717130235904)]} 0 3
[junit4:junit4] 2> 378300 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16]} 0 5
[junit4:junit4] 2> 378303 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17 (1438288717136527360)]} 0 0
[junit4:junit4] 2> 378309 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!17 (1438288717140721664)]} 0 0
[junit4:junit4] 2> 378310 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!17 (1438288717140721664)]} 0 4
[junit4:junit4] 2> 378310 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17]} 0 5
[junit4:junit4] 2> 378313 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438288717147013120)]} 0 0
[junit4:junit4] 2> 378320 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!18 (1438288717152256000)]} 0 0
[junit4:junit4] 2> 378321 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[s!18 (1438288717152256000)]} 0 3
[junit4:junit4] 2> 378321 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18]} 0 5
[junit4:junit4] 2> 378324 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19 (1438288717158547456)]} 0 0
[junit4:junit4] 2> 378329 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!19 (1438288717161693184)]} 0 0
[junit4:junit4] 2> 378330 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19 (1438288717161693184)]} 0 3
[junit4:junit4] 2> 378333 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20 (1438288717167984640)]} 0 0
[junit4:junit4] 2> 378340 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!20 (1438288717173227520)]} 0 0
[junit4:junit4] 2> 378341 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!20 (1438288717173227520)]} 0 3
[junit4:junit4] 2> 378341 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20]} 0 5
[junit4:junit4] 2> 378344 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21 (1438288717179518976)]} 0 0
[junit4:junit4] 2> 378351 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!21 (1438288717184761856)]} 0 0
[junit4:junit4] 2> 378352 T949 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!21 (1438288717184761856)]} 0 3
[junit4:junit4] 2> 378352 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21]} 0 5
[junit4:junit4] 2> 378355 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438288717191053312)]} 0 0
[junit4:junit4] 2> 378360 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!22 (1438288717194199040)]} 0 0
[junit4:junit4] 2> 378361 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438288717194199040)]} 0 3
[junit4:junit4] 2> 378363 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23 (1438288717199441920)]} 0 0
[junit4:junit4] 2> 378370 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!23 (1438288717204684800)]} 0 0
[junit4:junit4] 2> 378371 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!23 (1438288717204684800)]} 0 3
[junit4:junit4] 2> 378371 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23]} 0 5
[junit4:junit4] 2> 378374 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438288717210976256)]} 0 0
[junit4:junit4] 2> 378379 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!24 (1438288717214121984)]} 0 0
[junit4:junit4] 2> 378380 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438288717214121984)]} 0 3
[junit4:junit4] 2> 378383 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438288717220413440)]} 0 0
[junit4:junit4] 2> 378390 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!25 (1438288717225656320)]} 0 0
[junit4:junit4] 2> 378391 T949 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[z!25 (1438288717225656320)]} 0 3
[junit4:junit4] 2> 378391 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25]} 0 5
[junit4:junit4] 2> 378394 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26 (1438288717231947776)]} 0 0
[junit4:junit4] 2> 378401 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!26 (1438288717237190656)]} 0 0
[junit4:junit4] 2> 378402 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!26 (1438288717237190656)]} 0 3
[junit4:junit4] 2> 378402 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26]} 0 5
[junit4:junit4] 2> 378405 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27 (1438288717243482112)]} 0 0
[junit4:junit4] 2> 378413 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!27 (1438288717248724992)]} 0 0
[junit4:junit4] 2> 378414 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!27 (1438288717248724992)]} 0 4
[junit4:junit4] 2> 378415 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27]} 0 7
[junit4:junit4] 2> 378417 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438288717256065024)]} 0 0
[junit4:junit4] 2> 378422 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!28 (1438288717259210752)]} 0 0
[junit4:junit4] 2> 378423 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438288717259210752)]} 0 3
[junit4:junit4] 2> 378426 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438288717265502208)]} 0 0
[junit4:junit4] 2> 378433 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!29 (1438288717270745088)]} 0 0
[junit4:junit4] 2> 378434 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!29 (1438288717270745088)]} 0 3
[junit4:junit4] 2> 378434 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29]} 0 5
[junit4:junit4] 2> 378437 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30 (1438288717277036544)]} 0 0
[junit4:junit4] 2> 378444 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!30 (1438288717282279424)]} 0 0
[junit4:junit4] 2> 378445 T961 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!30 (1438288717282279424)]} 0 3
[junit4:junit4] 2> 378445 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30]} 0 5
[junit4:junit4] 2> 378448 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31 (1438288717288570880)]} 0 0
[junit4:junit4] 2> 378455 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!31 (1438288717293813760)]} 0 0
[junit4:junit4] 2> 378456 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!31 (1438288717293813760)]} 0 3
[junit4:junit4] 2> 378456 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31]} 0 5
[junit4:junit4] 2> 378459 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32 (1438288717300105216)]} 0 0
[junit4:junit4] 2> 378466 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!32 (1438288717305348096)]} 0 0
[junit4:junit4] 2> 378467 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!32 (1438288717305348096)]} 0 3
[junit4:junit4] 2> 378467 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32]} 0 5
[junit4:junit4] 2> 378470 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33 (1438288717311639552)]} 0 0
[junit4:junit4] 2> 378477 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!33 (1438288717316882432)]} 0 0
[junit4:junit4] 2> 378478 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!33 (1438288717316882432)]} 0 3
[junit4:junit4] 2> 378478 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33]} 0 5
[junit4:junit4] 2> 378481 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34 (1438288717323173888)]} 0 0
[junit4:junit4] 2> 378488 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!34 (1438288717328416768)]} 0 0
[junit4:junit4] 2> 378489 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!34 (1438288717328416768)]} 0 3
[junit4:junit4] 2> 378489 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34]} 0 5
[junit4:junit4] 2> 378492 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438288717334708224)]} 0 0
[junit4:junit4] 2> 378497 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!35 (1438288717337853952)]} 0 0
[junit4:junit4] 2> 378498 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438288717337853952)]} 0 3
[junit4:junit4] 2> 378500 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36 (1438288717343096832)]} 0 0
[junit4:junit4] 2> 378507 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!36 (1438288717348339712)]} 0 0
[junit4:junit4] 2> 378508 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!36 (1438288717348339712)]} 0 3
[junit4:junit4] 2> 378508 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36]} 0 5
[junit4:junit4] 2> 378511 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37 (1438288717354631168)]} 0 0
[junit4:junit4] 2> 378518 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!37 (1438288717359874048)]} 0 0
[junit4:junit4] 2> 378519 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!37 (1438288717359874048)]} 0 3
[junit4:junit4] 2> 378520 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37]} 0 6
[junit4:junit4] 2> 378522 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438288717366165504)]} 0 0
[junit4:junit4] 2> 378527 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!38 (1438288717369311232)]} 0 0
[junit4:junit4] 2> 378528 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438288717369311232)]} 0 3
[junit4:junit4] 2> 378531 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39 (1438288717375602688)]} 0 0
[junit4:junit4] 2> 378538 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!39 (1438288717380845568)]} 0 0
[junit4:junit4] 2> 378539 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!39 (1438288717380845568)]} 0 3
[junit4:junit4] 2> 378539 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39]} 0 5
[junit4:junit4] 2> 378542 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40 (1438288717387137024)]} 0 0
[junit4:junit4] 2> 378549 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!40 (1438288717392379904)]} 0 0
[junit4:junit4] 2> 378550 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!40 (1438288717392379904)]} 0 3
[junit4:junit4] 2> 378550 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40]} 0 5
[junit4:junit4] 2> 378553 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41 (1438288717398671360)]} 0 0
[junit4:junit4] 2> 378560 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!41 (1438288717403914240)]} 0 0
[junit4:junit4] 2> 378561 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!41 (1438288717403914240)]} 0 3
[junit4:junit4] 2> 378561 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41]} 0 5
[junit4:junit4] 2> 378564 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42 (1438288717410205696)]} 0 0
[junit4:junit4] 2> 378572 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!42 (1438288717415448576)]} 0 0
[junit4:junit4] 2> 378573 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!42 (1438288717415448576)]} 0 4
[junit4:junit4] 2> 378574 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42]} 0 7
[junit4:junit4] 2> 378576 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43 (1438288717422788608)]} 0 0
[junit4:junit4] 2> 378584 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!43 (1438288717428031488)]} 0 0
[junit4:junit4] 2> 378585 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!43 (1438288717428031488)]} 0 4
[junit4:junit4] 2> 378586 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43]} 0 7
[junit4:junit4] 2> 378588 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44 (1438288717435371520)]} 0 0
[junit4:junit4] 2> 378593 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!44 (1438288717438517248)]} 0 0
[junit4:junit4] 2> 378594 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44 (1438288717438517248)]} 0 3
[junit4:junit4] 2> 378597 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45 (1438288717444808704)]} 0 0
[junit4:junit4] 2> 378604 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!45 (1438288717450051584)]} 0 0
[junit4:junit4] 2> 378605 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!45 (1438288717450051584)]} 0 3
[junit4:junit4] 2> 378605 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45]} 0 5
[junit4:junit4] 2> 378608 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46 (1438288717456343040)]} 0 0
[junit4:junit4] 2> 378615 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!46 (1438288717461585920)]} 0 0
[junit4:junit4] 2> 378616 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!46 (1438288717461585920)]} 0 3
[junit4:junit4] 2> 378617 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46]} 0 5
[junit4:junit4] 2> 378619 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47 (1438288717467877376)]} 0 0
[junit4:junit4] 2> 378626 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!47 (1438288717473120256)]} 0 0
[junit4:junit4] 2> 378627 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!47 (1438288717473120256)]} 0 3
[junit4:junit4] 2> 378627 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47]} 0 5
[junit4:junit4] 2> 378630 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48 (1438288717479411712)]} 0 0
[junit4:junit4] 2> 378637 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!48 (1438288717484654592)]} 0 0
[junit4:junit4] 2> 378638 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!48 (1438288717484654592)]} 0 3
[junit4:junit4] 2> 378638 T996 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48]} 0 5
[junit4:junit4] 2> 378641 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49 (1438288717490946048)]} 0 0
[junit4:junit4] 2> 378646 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!49 (1438288717494091776)]} 0 0
[junit4:junit4] 2> 378647 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49 (1438288717494091776)]} 0 3
[junit4:junit4] 2> 378650 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438288717500383232)]} 0 0
[junit4:junit4] 2> 378655 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!50 (1438288717503528960)]} 0 0
[junit4:junit4] 2> 378656 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438288717503528960)]} 0 3
[junit4:junit4] 2> 378658 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51 (1438288717508771840)]} 0 0
[junit4:junit4] 2> 378665 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!51 (1438288717514014720)]} 0 0
[junit4:junit4] 2> 378666 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[z!51 (1438288717514014720)]} 0 3
[junit4:junit4] 2> 378666 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51]} 0 5
[junit4:junit4] 2> 378669 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52 (1438288717520306176)]} 0 0
[junit4:junit4] 2> 378676 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!52 (1438288717525549056)]} 0 0
[junit4:junit4] 2> 378677 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!52 (1438288717525549056)]} 0 3
[junit4:junit4] 2> 378677 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52]} 0 5
[junit4:junit4] 2> 378680 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53 (1438288717531840512)]} 0 0
[junit4:junit4] 2> 378685 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!53 (1438288717534986240)]} 0 0
[junit4:junit4] 2> 378686 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53 (1438288717534986240)]} 0 3
[junit4:junit4] 2> 378689 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54 (1438288717541277696)]} 0 0
[junit4:junit4] 2> 378696 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!54 (1438288717546520576)]} 0 0
[junit4:junit4] 2> 378697 T945 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!54 (1438288717546520576)]} 0 3
[junit4:junit4] 2> 378698 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54]} 0 5
[junit4:junit4] 2> 378700 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55 (1438288717552812032)]} 0 0
[junit4:junit4] 2> 378707 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!55 (1438288717558054912)]} 0 0
[junit4:junit4] 2> 378708 T963 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:53094/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!55 (1438288717558054912)]} 0 3
[junit4:junit4] 2> 378708 T980 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55]} 0 5
[junit4:junit4] 2> 378711 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56 (1438288717564346368)]} 0 0
[junit4:junit4] 2> 378716 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!56 (1438288717567492096)]} 0 0
[junit4:junit4] 2> 378717 T965 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56 (1438288717567492096)]} 0 3
[junit4:junit4] 2> 378719 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57 (1438288717572734976)]} 0 0
[junit4:junit4] 2> 378726 T994 C1728 P20360 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11776/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!57 (1438288717577977856)]} 0 0
[junit4:junit4] 2> 378727 T1021 C1729 P11776 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!57 (1438288717577977856)]} 0 3
[junit4:junit4] 2> 378727 T947 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57]} 0 5
[junit4:junit4] 2> 378730 T925 C1730 P43958 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!58 (1438288717584269312)]} 0 0
[junit4:junit4] 2> 378737 T979 C1726 P53094 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:29823/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!58 (1438288717589512192)]} 0 0
[junit4:junit4] 2> 378738 T951 C1727 P29823 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20360/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!58 (1438288717589512192)]} 0 3
[junit4:junit4] 2> 378738 T996 C1728 P20360 oasup.LogUpdateP
[...truncated too long message...]
unit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:56622_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56622"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:10320_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:10320",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:31872_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:56622_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:10320_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:38692_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893085820682254-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893085820682252-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893085820682250-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:31872_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:31872"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89893085820682243-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:10320_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:10320"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89893085820682254-127.0.0.1:38692_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893085820682243-127.0.0.1:10320_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893085820682252-127.0.0.1:56622_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893085820682250-127.0.0.1:31872_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89893085820682243-127.0.0.1:10320_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=BA912B0BBB7987CE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=Etc/GMT+5 -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 839s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:38692/collection1lastClient and got 234 from http://127.0.0.1:31872/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([BA912B0BBB7987CE:3B77A513CC26E7F2]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1843037 T2094 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 838811 T2093 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 1844090 T2186 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1844090 T2186 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1844090 T2186 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=hr_HR, timezone=Etc/GMT+5
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=307819912,total=462422016
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestSystemIdResolver, PingRequestHandlerTest, TestSolrJ, TestIBSimilarityFactory, TestMergePolicyConfig, TestMaxScoreQueryParser, TestRemoteStreaming, DefaultValueUpdateProcessorTest, IndexSchemaRuntimeFieldTest, WordBreakSolrSpellCheckerTest, SliceStateUpdateTest, TestBinaryResponseWriter, TestFieldTypeResource, UUIDFieldTest, TermVectorComponentTest, TestUniqueKeyFieldResource, LoggingHandlerTest, SynonymTokenizerTest, DistributedQueryElevationComponentTest, TestStressLucene, AnalysisAfterCoreReloadTest, XmlUpdateRequestHandlerTest, PluginInfoTest, TestDocumentBuilder, DocValuesMultiTest, PrimUtilsTest, SolrIndexConfigTest, TestSolr4Spatial, SuggesterTSTTest, TestLRUCache, TestFastOutputStream, QueryEqualityTest, CollectionsAPIDistributedZkTest, TestSort, JSONWriterTest, TestPropInject, LegacyHTMLStripCharFilterTest, FullSolrCloudDistribCmdsTest, TestSolrIndexConfig, BasicDistributedZk2Test, TestCloudManagedSchema, TestSuggestSpellingConverter, SearchHandlerTest, TestSchemaSimilarityResource, StatelessScriptUpdateProcessorFactoryTest, TestXIncludeConfig, SpatialFilterTest, BasicDistributedZkTest, BasicFunctionalityTest, CopyFieldTest, ResourceLoaderTest, TestFiltering, TestPHPSerializedResponseWriter, TestFieldCollectionResource, ZkSolrClientTest, SignatureUpdateProcessorFactoryTest, TestFastLRUCache, TestSerializedLuceneMatchVersion, PolyFieldTest, DocumentBuilderTest, TestUpdate, DebugComponentTest, TestCollationField, ReturnFieldsTest, ZkCLITest, TestCollationKeyRangeQueries, DirectSolrSpellCheckerTest, NoCacheHeaderTest, SOLR749Test, TestLMDirichletSimilarityFactory, TestMultiCoreConfBootstrap, ChaosMonkeyShardSplitTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 840.10s, 1 test, 1 failure <<< FAILURES!
[...truncated 425 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1246: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:890: There were test failures: 298 suites, 1253 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 57 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure