You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/12 20:10:18 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1326 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1326/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:34833 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:34833 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([D53082544D8F9C02:54D60C4C3AD0FC3E]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9640 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 1044413 T2135 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1044419 T2135 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1371059723866
[junit4:junit4] 2> 1044421 T2135 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1044422 T2136 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1044523 T2135 oasc.ZkTestServer.run start zk server on port:63758
[junit4:junit4] 2> 1044524 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1044529 T2142 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9e6032d name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1044529 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1044530 T2135 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1044565 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1044567 T2144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@721fbe33 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1044568 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1044568 T2135 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1044581 T2135 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1044584 T2135 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1044586 T2135 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1044595 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1044596 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1044602 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1044603 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1044707 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1044708 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1044711 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1044711 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1044740 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1044741 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1044744 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1044745 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1044752 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1044753 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1044756 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1044757 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1044760 T2135 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1044760 T2135 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1045026 T2135 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1045032 T2135 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35898
[junit4:junit4] 2> 1045032 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1045033 T2135 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1045033 T2135 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209
[junit4:junit4] 2> 1045034 T2135 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/solr.xml
[junit4:junit4] 2> 1045034 T2135 oasc.CoreContainer.<init> New CoreContainer 1854023765
[junit4:junit4] 2> 1045035 T2135 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/'
[junit4:junit4] 2> 1045036 T2135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/'
[junit4:junit4] 2> 1045162 T2135 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1045162 T2135 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1045163 T2135 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1045164 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1045164 T2135 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1045165 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1045165 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1045166 T2135 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1045166 T2135 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1045167 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1045184 T2135 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1045185 T2135 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63758/solr
[junit4:junit4] 2> 1045186 T2135 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1045187 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1045189 T2155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63b2296 name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1045190 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1045192 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1045205 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1045207 T2157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ed55e5 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1045207 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1045209 T2135 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1045219 T2135 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1045222 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1045229 T2135 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35898_
[junit4:junit4] 2> 1045231 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35898_
[junit4:junit4] 2> 1045238 T2135 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1045252 T2135 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1045254 T2135 oasc.Overseer.start Overseer (id=89853770063740931-127.0.0.1:35898_-n_0000000000) starting
[junit4:junit4] 2> 1045264 T2135 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1045274 T2159 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1045274 T2135 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1045282 T2135 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1045284 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1045286 T2158 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1045290 T2160 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1045290 T2160 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1046789 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1046790 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35898_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35898"}
[junit4:junit4] 2> 1046790 T2158 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1046791 T2158 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1046830 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1047294 T2160 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/collection1
[junit4:junit4] 2> 1047294 T2160 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1047295 T2160 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1047295 T2160 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1047296 T2160 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/collection1/'
[junit4:junit4] 2> 1047298 T2160 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/collection1/lib/README' to classloader
[junit4:junit4] 2> 1047298 T2160 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1047350 T2160 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1047410 T2160 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1047511 T2160 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1047516 T2160 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1048082 T2160 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1048086 T2160 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1048088 T2160 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1048093 T2160 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1048114 T2160 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1048114 T2160 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371059724209/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data/
[junit4:junit4] 2> 1048114 T2160 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e47c8ac
[junit4:junit4] 2> 1048115 T2160 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1048115 T2160 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data
[junit4:junit4] 2> 1048116 T2160 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data/index/
[junit4:junit4] 2> 1048116 T2160 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1048116 T2160 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data/index
[junit4:junit4] 2> 1048121 T2160 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b52695b lockFactory=org.apache.lucene.store.NativeFSLockFactory@18c941e7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1048121 T2160 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1048124 T2160 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1048124 T2160 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1048125 T2160 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1048125 T2160 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1048126 T2160 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1048126 T2160 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1048126 T2160 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1048127 T2160 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1048127 T2160 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1048142 T2160 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1048149 T2160 oass.SolrIndexSearcher.<init> Opening Searcher@6bbf85a main
[junit4:junit4] 2> 1048150 T2160 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/control/data/tlog
[junit4:junit4] 2> 1048151 T2160 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1048151 T2160 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1048155 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6bbf85a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1048156 T2160 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1048156 T2160 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35898 collection:control_collection shard:shard1
[junit4:junit4] 2> 1048157 T2160 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1048175 T2160 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1048187 T2160 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1048187 T2160 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1048188 T2160 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35898/collection1/
[junit4:junit4] 2> 1048188 T2160 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1048188 T2160 oasc.SyncStrategy.syncToMe http://127.0.0.1:35898/collection1/ has no replicas
[junit4:junit4] 2> 1048188 T2160 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35898/collection1/
[junit4:junit4] 2> 1048189 T2160 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1048334 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1048349 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1048406 T2160 oasc.ZkController.register We are http://127.0.0.1:35898/collection1/ and leader is http://127.0.0.1:35898/collection1/
[junit4:junit4] 2> 1048406 T2160 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35898
[junit4:junit4] 2> 1048406 T2160 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1048406 T2160 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1048407 T2160 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1048409 T2160 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1048410 T2135 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1048410 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1048411 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1048427 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1048429 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1048431 T2164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d99d761 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1048431 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1048432 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1048434 T2135 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1048695 T2135 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1048699 T2135 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34833
[junit4:junit4] 2> 1048700 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1048700 T2135 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1048701 T2135 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880
[junit4:junit4] 2> 1048701 T2135 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/solr.xml
[junit4:junit4] 2> 1048702 T2135 oasc.CoreContainer.<init> New CoreContainer 1726301600
[junit4:junit4] 2> 1048703 T2135 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/'
[junit4:junit4] 2> 1048703 T2135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/'
[junit4:junit4] 2> 1048829 T2135 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1048830 T2135 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1048831 T2135 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1048831 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1048832 T2135 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1048832 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1048833 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1048833 T2135 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1048834 T2135 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1048834 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1048851 T2135 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1048852 T2135 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63758/solr
[junit4:junit4] 2> 1048853 T2135 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1048854 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1048857 T2175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46051d99 name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1048858 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1048872 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1048884 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1048886 T2177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b9cbc87 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1048886 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1048898 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1049864 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1049865 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35898_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35898"}
[junit4:junit4] 2> 1049879 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1049879 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1049879 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1049901 T2135 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34833_
[junit4:junit4] 2> 1049903 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34833_
[junit4:junit4] 2> 1049906 T2157 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1049906 T2177 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1049906 T2164 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1049907 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1049907 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1049907 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1049918 T2178 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1049919 T2178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1051383 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1051384 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34833_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34833"}
[junit4:junit4] 2> 1051384 T2158 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1051385 T2158 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1051402 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1051402 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1051402 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1051922 T2178 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/collection1
[junit4:junit4] 2> 1051922 T2178 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1051923 T2178 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1051923 T2178 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1051925 T2178 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/collection1/'
[junit4:junit4] 2> 1051926 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1051927 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/collection1/lib/README' to classloader
[junit4:junit4] 2> 1051978 T2178 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1052038 T2178 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1052139 T2178 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1052145 T2178 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1052692 T2178 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1052696 T2178 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1052698 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1052703 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1052724 T2178 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1052724 T2178 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371059727880/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1/
[junit4:junit4] 2> 1052724 T2178 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e47c8ac
[junit4:junit4] 2> 1052725 T2178 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1052726 T2178 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1
[junit4:junit4] 2> 1052726 T2178 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1/index/
[junit4:junit4] 2> 1052726 T2178 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1052727 T2178 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1/index
[junit4:junit4] 2> 1052731 T2178 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1052732 T2178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1052735 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1052735 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1052736 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1052737 T2178 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1052737 T2178 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1052737 T2178 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1052738 T2178 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1052738 T2178 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1052739 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1052755 T2178 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1052763 T2178 oass.SolrIndexSearcher.<init> Opening Searcher@744a4b85 main
[junit4:junit4] 2> 1052763 T2178 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty1/tlog
[junit4:junit4] 2> 1052764 T2178 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1052765 T2178 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1052769 T2179 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@744a4b85 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1052771 T2178 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1052772 T2178 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34833 collection:collection1 shard:shard1
[junit4:junit4] 2> 1052772 T2178 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1052792 T2178 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1052802 T2178 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1052802 T2178 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1052803 T2178 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34833/collection1/
[junit4:junit4] 2> 1052803 T2178 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1052803 T2178 oasc.SyncStrategy.syncToMe http://127.0.0.1:34833/collection1/ has no replicas
[junit4:junit4] 2> 1052803 T2178 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34833/collection1/
[junit4:junit4] 2> 1052804 T2178 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1052906 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1052921 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1052921 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1052921 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1052968 T2178 oasc.ZkController.register We are http://127.0.0.1:34833/collection1/ and leader is http://127.0.0.1:34833/collection1/
[junit4:junit4] 2> 1052968 T2178 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34833
[junit4:junit4] 2> 1052968 T2178 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1052968 T2178 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1052969 T2178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1052970 T2178 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1052971 T2135 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1052972 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1052973 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1053253 T2135 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1053257 T2135 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:20054
[junit4:junit4] 2> 1053257 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1053258 T2135 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1053258 T2135 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435
[junit4:junit4] 2> 1053259 T2135 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/solr.xml
[junit4:junit4] 2> 1053259 T2135 oasc.CoreContainer.<init> New CoreContainer 531521047
[junit4:junit4] 2> 1053260 T2135 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/'
[junit4:junit4] 2> 1053261 T2135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/'
[junit4:junit4] 2> 1053387 T2135 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1053388 T2135 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1053389 T2135 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1053389 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1053390 T2135 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1053390 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1053391 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1053391 T2135 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1053392 T2135 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1053392 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1053410 T2135 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1053410 T2135 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63758/solr
[junit4:junit4] 2> 1053411 T2135 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1053412 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1053415 T2191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c208206 name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1053416 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1053430 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1053442 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1053444 T2193 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c7c4bd3 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1053444 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1053449 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1054426 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1054427 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34833_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34833"}
[junit4:junit4] 2> 1054443 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1054443 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1054443 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1054443 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1054452 T2135 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:20054_
[junit4:junit4] 2> 1054454 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:20054_
[junit4:junit4] 2> 1054456 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1054456 T2157 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1054457 T2164 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1054456 T2177 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1054457 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1054457 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1054457 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1054458 T2193 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1054486 T2194 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1054486 T2194 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1055947 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1055948 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20054_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20054"}
[junit4:junit4] 2> 1055949 T2158 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1055949 T2158 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1056040 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1056040 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1056040 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1056040 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1056491 T2194 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/collection1
[junit4:junit4] 2> 1056491 T2194 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1056492 T2194 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1056492 T2194 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1056493 T2194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/collection1/'
[junit4:junit4] 2> 1056495 T2194 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1056495 T2194 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/collection1/lib/README' to classloader
[junit4:junit4] 2> 1056547 T2194 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1056606 T2194 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1056707 T2194 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1056713 T2194 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1057258 T2194 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1057262 T2194 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1057265 T2194 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1057270 T2194 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1057290 T2194 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1057290 T2194 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371059732435/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2/
[junit4:junit4] 2> 1057291 T2194 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e47c8ac
[junit4:junit4] 2> 1057291 T2194 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1057292 T2194 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2
[junit4:junit4] 2> 1057292 T2194 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2/index/
[junit4:junit4] 2> 1057293 T2194 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1057294 T2194 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2/index
[junit4:junit4] 2> 1057297 T2194 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1057298 T2194 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1057301 T2194 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1057301 T2194 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1057302 T2194 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1057303 T2194 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1057303 T2194 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1057304 T2194 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1057304 T2194 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1057305 T2194 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1057305 T2194 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1057321 T2194 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1057329 T2194 oass.SolrIndexSearcher.<init> Opening Searcher@316ccc38 main
[junit4:junit4] 2> 1057329 T2194 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty2/tlog
[junit4:junit4] 2> 1057330 T2194 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1057330 T2194 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1057335 T2195 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@316ccc38 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1057338 T2194 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1057338 T2194 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:20054 collection:collection1 shard:shard2
[junit4:junit4] 2> 1057339 T2194 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1057367 T2194 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1057379 T2194 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1057380 T2194 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1057380 T2194 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:20054/collection1/
[junit4:junit4] 2> 1057380 T2194 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1057380 T2194 oasc.SyncStrategy.syncToMe http://127.0.0.1:20054/collection1/ has no replicas
[junit4:junit4] 2> 1057381 T2194 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:20054/collection1/
[junit4:junit4] 2> 1057381 T2194 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1057593 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1057616 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1057616 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1057616 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1057616 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1057650 T2194 oasc.ZkController.register We are http://127.0.0.1:20054/collection1/ and leader is http://127.0.0.1:20054/collection1/
[junit4:junit4] 2> 1057650 T2194 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:20054
[junit4:junit4] 2> 1057650 T2194 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1057650 T2194 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1057651 T2194 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1057653 T2194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1057654 T2135 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1057655 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1057655 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1057938 T2135 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1057942 T2135 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:29467
[junit4:junit4] 2> 1057943 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1057943 T2135 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1057944 T2135 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118
[junit4:junit4] 2> 1057944 T2135 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/solr.xml
[junit4:junit4] 2> 1057945 T2135 oasc.CoreContainer.<init> New CoreContainer 1125258243
[junit4:junit4] 2> 1057945 T2135 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/'
[junit4:junit4] 2> 1057946 T2135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/'
[junit4:junit4] 2> 1058072 T2135 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1058073 T2135 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1058073 T2135 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1058074 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1058074 T2135 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1058075 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1058075 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1058076 T2135 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1058076 T2135 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1058077 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1058094 T2135 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1058095 T2135 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63758/solr
[junit4:junit4] 2> 1058095 T2135 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1058096 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1058099 T2207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23e7afe6 name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1058100 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1058118 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1058130 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1058133 T2209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@138a0886 name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1058133 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1058143 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1059130 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1059131 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20054_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20054"}
[junit4:junit4] 2> 1059148 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059149 T2135 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:29467_
[junit4:junit4] 2> 1059148 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059148 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059148 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059148 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059157 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:29467_
[junit4:junit4] 2> 1059160 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1059160 T2177 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1059160 T2209 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1059160 T2164 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1059160 T2157 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1059161 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1059161 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1059160 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1059161 T2193 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1059161 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1059168 T2210 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1059169 T2210 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1060660 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1060661 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29467"}
[junit4:junit4] 2> 1060661 T2158 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1060662 T2158 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1060678 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1060678 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1060678 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1060678 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1060678 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1061172 T2210 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/collection1
[junit4:junit4] 2> 1061172 T2210 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1061173 T2210 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1061173 T2210 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1061175 T2210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/collection1/'
[junit4:junit4] 2> 1061177 T2210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1061177 T2210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/collection1/lib/README' to classloader
[junit4:junit4] 2> 1061236 T2210 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1061304 T2210 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1061405 T2210 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1061411 T2210 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1062042 T2210 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1062047 T2210 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1062049 T2210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1062055 T2210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1062073 T2210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1062074 T2210 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371059737118/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/
[junit4:junit4] 2> 1062074 T2210 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e47c8ac
[junit4:junit4] 2> 1062075 T2210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1062076 T2210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3
[junit4:junit4] 2> 1062077 T2210 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index/
[junit4:junit4] 2> 1062077 T2210 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1062077 T2210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index
[junit4:junit4] 2> 1062081 T2210 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64da74f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d405676),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1062082 T2210 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1062085 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1062085 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1062086 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1062087 T2210 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1062087 T2210 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1062088 T2210 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1062088 T2210 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1062089 T2210 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1062089 T2210 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1062109 T2210 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1062117 T2210 oass.SolrIndexSearcher.<init> Opening Searcher@b56dcd0 main
[junit4:junit4] 2> 1062118 T2210 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/tlog
[junit4:junit4] 2> 1062119 T2210 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1062119 T2210 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1062123 T2211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b56dcd0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1062126 T2210 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1062126 T2210 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:29467 collection:collection1 shard:shard1
[junit4:junit4] 2> 1062130 T2210 oasc.ZkController.register We are http://127.0.0.1:29467/collection1/ and leader is http://127.0.0.1:34833/collection1/
[junit4:junit4] 2> 1062130 T2210 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:29467
[junit4:junit4] 2> 1062131 T2210 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1062131 T2210 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C381 name=collection1 org.apache.solr.core.SolrCore@7667c83e url=http://127.0.0.1:29467/collection1 node=127.0.0.1:29467_ C381_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:29467_, base_url=http://127.0.0.1:29467}
[junit4:junit4] 2> 1062131 T2212 C381 P29467 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1062132 T2210 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1062133 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1062133 T2212 C381 P29467 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1062133 T2212 C381 P29467 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1062134 T2135 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1062134 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1062135 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1062135 T2212 C381 P29467 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1062153 T2167 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1062182 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1062183 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29467"}
[junit4:junit4] 2> 1062203 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1062203 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1062203 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1062203 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1062203 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1062331 T2135 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1062334 T2135 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25676
[junit4:junit4] 2> 1062334 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1062335 T2135 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1062335 T2135 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596
[junit4:junit4] 2> 1062335 T2135 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/solr.xml
[junit4:junit4] 2> 1062336 T2135 oasc.CoreContainer.<init> New CoreContainer 1390355055
[junit4:junit4] 2> 1062336 T2135 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/'
[junit4:junit4] 2> 1062336 T2135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/'
[junit4:junit4] 2> 1062433 T2135 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1062433 T2135 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1062434 T2135 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1062434 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1062435 T2135 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1062435 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1062435 T2135 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1062436 T2135 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1062436 T2135 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1062436 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1062449 T2135 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1062450 T2135 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63758/solr
[junit4:junit4] 2> 1062450 T2135 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1062451 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1062453 T2224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66914706 name:ZooKeeperConnection Watcher:127.0.0.1:63758 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1062454 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1062455 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1062463 T2135 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1062465 T2226 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@104773ff name:ZooKeeperConnection Watcher:127.0.0.1:63758/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1062466 T2135 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1062477 T2135 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1063154 T2167 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1063154 T2167 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:29467_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 1063479 T2135 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25676_
[junit4:junit4] 2> 1063481 T2135 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25676_
[junit4:junit4] 2> 1063494 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1063494 T2157 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063494 T2226 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063495 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063494 T2209 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063494 T2177 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063494 T2164 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063495 T2193 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1063495 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063495 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063495 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063501 T2227 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1063501 T2227 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1063707 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1063708 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25676_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25676"}
[junit4:junit4] 2> 1063708 T2158 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1063708 T2158 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1063721 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063721 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063721 T2226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063721 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063721 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1063721 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1064504 T2227 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/collection1
[junit4:junit4] 2> 1064504 T2227 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1064505 T2227 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1064505 T2227 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1064506 T2227 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/collection1/'
[junit4:junit4] 2> 1064508 T2227 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/collection1/lib/README' to classloader
[junit4:junit4] 2> 1064508 T2227 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1064561 T2227 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1064620 T2227 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1064722 T2227 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1064728 T2227 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C381_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:29467_, base_url=http://127.0.0.1:29467}
[junit4:junit4] 2> 1065156 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34833/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1065156 T2212 C381 P29467 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:29467 START replicas=[http://127.0.0.1:34833/collection1/] nUpdates=100
[junit4:junit4] 2> 1065157 T2212 C381 P29467 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1065158 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1065158 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1065158 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1065158 T2212 C381 P29467 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1065159 T2212 C381 P29467 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34833/collection1/. core=collection1
[junit4:junit4] 2> 1065159 T2212 C381 P29467 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C382 name=collection1 org.apache.solr.core.SolrCore@2a5798fd url=http://127.0.0.1:34833/collection1 node=127.0.0.1:34833_ C382_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34833_, base_url=http://127.0.0.1:34833, leader=true}
[junit4:junit4] 2> 1065167 T2168 C382 P34833 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1065172 T2169 C382 P34833 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1065175 T2169 C382 P34833 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1065176 T2169 C382 P34833 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1065176 T2169 C382 P34833 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1065177 T2169 C382 P34833 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1065178 T2169 C382 P34833 oass.SolrIndexSearcher.<init> Opening Searcher@184f4072 realtime
[junit4:junit4] 2> 1065178 T2169 C382 P34833 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1065178 T2169 C382 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1065179 T2212 C381 P29467 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1065180 T2212 C381 P29467 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1065182 T2170 C382 P34833 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1065182 T2170 C382 P34833 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1065183 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1065183 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1065183 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1065186 T2170 C382 P34833 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1065187 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1065188 T2212 C381 P29467 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index.20130612135544632
[junit4:junit4] 2> 1065188 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@627094f5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b079148) fullCopy=false
[junit4:junit4] 2> 1065191 T2170 C382 P34833 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1065192 T2212 C381 P29467 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1065193 T2212 C381 P29467 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1065194 T2212 C381 P29467 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1065196 T2212 C381 P29467 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64da74f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d405676),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64da74f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d405676),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1065196 T2212 C381 P29467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1065197 T2212 C381 P29467 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1065197 T2212 C381 P29467 oass.SolrIndexSearcher.<init> Opening Searcher@35e533fd main
[junit4:junit4] 2> 1065198 T2211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35e533fd main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1065199 T2212 C381 P29467 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index.20130612135544632 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index.20130612135544632;done=true>>]
[junit4:junit4] 2> 1065199 T2212 C381 P29467 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index.20130612135544632
[junit4:junit4] 2> 1065199 T2212 C381 P29467 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty3/index.20130612135544632
[junit4:junit4] 2> 1065200 T2212 C381 P29467 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1065200 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1065200 T2212 C381 P29467 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1065200 T2212 C381 P29467 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1065202 T2212 C381 P29467 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1065225 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1065226 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29467"}
[junit4:junit4] 2> 1065238 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065238 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065238 T2226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065238 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065238 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065238 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1065296 T2227 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1065300 T2227 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1065303 T2227 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1065308 T2227 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1065328 T2227 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1065329 T2227 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371059741596/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/
[junit4:junit4] 2> 1065329 T2227 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e47c8ac
[junit4:junit4] 2> 1065330 T2227 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1065331 T2227 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4
[junit4:junit4] 2> 1065331 T2227 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index/
[junit4:junit4] 2> 1065331 T2227 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1065332 T2227 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index
[junit4:junit4] 2> 1065336 T2227 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50e1d4fa lockFactory=org.apache.lucene.store.NativeFSLockFactory@658efbd),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1065336 T2227 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1065339 T2227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1065340 T2227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1065340 T2227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1065341 T2227 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1065342 T2227 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1065342 T2227 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1065343 T2227 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1065344 T2227 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1065344 T2227 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1065360 T2227 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1065368 T2227 oass.SolrIndexSearcher.<init> Opening Searcher@7de155e4 main
[junit4:junit4] 2> 1065368 T2227 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/tlog
[junit4:junit4] 2> 1065369 T2227 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1065370 T2227 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1065375 T2230 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7de155e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1065377 T2227 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1065377 T2227 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25676 collection:collection1 shard:shard2
[junit4:junit4] 2> 1065381 T2227 oasc.ZkController.register We are http://127.0.0.1:25676/collection1/ and leader is http://127.0.0.1:20054/collection1/
[junit4:junit4] 2> 1065381 T2227 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25676
[junit4:junit4] 2> 1065382 T2227 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1065382 T2227 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C383 name=collection1 org.apache.solr.core.SolrCore@e72490d url=http://127.0.0.1:25676/collection1 node=127.0.0.1:25676_ C383_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:25676_, base_url=http://127.0.0.1:25676}
[junit4:junit4] 2> 1065382 T2231 C383 P25676 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1065383 T2227 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1065383 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1065383 T2231 C383 P25676 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1065384 T2231 C383 P25676 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1065384 T2135 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1065385 T2135 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1065385 T2231 C383 P25676 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1065385 T2135 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1065411 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1065411 T2183 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1065412 T2135 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1065413 T2135 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1065413 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1066415 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1066742 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1066743 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25676_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25676"}
[junit4:junit4] 2> 1066748 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1066748 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1066748 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1066748 T2226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1066748 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1066748 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1067414 T2183 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1067414 T2183 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:25676_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1067417 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1068419 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C383_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:25676_, base_url=http://127.0.0.1:25676}
[junit4:junit4] 2> 1069416 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:20054/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1069416 T2231 C383 P25676 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:25676 START replicas=[http://127.0.0.1:20054/collection1/] nUpdates=100
[junit4:junit4] 2> 1069417 T2231 C383 P25676 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1069417 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1069417 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1069417 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1069417 T2231 C383 P25676 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1069418 T2231 C383 P25676 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:20054/collection1/. core=collection1
[junit4:junit4] 2> 1069418 T2231 C383 P25676 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1069421 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C384 name=collection1 org.apache.solr.core.SolrCore@32c8e49e url=http://127.0.0.1:20054/collection1 node=127.0.0.1:20054_ C384_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:20054_, base_url=http://127.0.0.1:20054, leader=true}
[junit4:junit4] 2> 1069426 T2184 C384 P20054 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1069430 T2185 C384 P20054 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1069434 T2185 C384 P20054 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1069435 T2185 C384 P20054 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1069435 T2185 C384 P20054 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1069436 T2185 C384 P20054 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1069437 T2185 C384 P20054 oass.SolrIndexSearcher.<init> Opening Searcher@3fa4a8d1 realtime
[junit4:junit4] 2> 1069437 T2185 C384 P20054 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1069437 T2185 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1069438 T2231 C383 P25676 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1069439 T2231 C383 P25676 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1069441 T2186 C384 P20054 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1069441 T2186 C384 P20054 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1069442 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1069442 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1069443 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1069445 T2186 C384 P20054 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1069446 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1069447 T2231 C383 P25676 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index.20130612135548891
[junit4:junit4] 2> 1069447 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@3da5d9b6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f8aaceb) fullCopy=false
[junit4:junit4] 2> 1069450 T2186 C384 P20054 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1069451 T2231 C383 P25676 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1069452 T2231 C383 P25676 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1069452 T2231 C383 P25676 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1069454 T2231 C383 P25676 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50e1d4fa lockFactory=org.apache.lucene.store.NativeFSLockFactory@658efbd),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50e1d4fa lockFactory=org.apache.lucene.store.NativeFSLockFactory@658efbd),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1069455 T2231 C383 P25676 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1069455 T2231 C383 P25676 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1069455 T2231 C383 P25676 oass.SolrIndexSearcher.<init> Opening Searcher@3ec28f1 main
[junit4:junit4] 2> 1069457 T2230 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ec28f1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1069457 T2231 C383 P25676 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index.20130612135548891 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index.20130612135548891;done=true>>]
[junit4:junit4] 2> 1069458 T2231 C383 P25676 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index.20130612135548891
[junit4:junit4] 2> 1069458 T2231 C383 P25676 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index.20130612135548891
[junit4:junit4] 2> 1069458 T2231 C383 P25676 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1069458 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1069459 T2231 C383 P25676 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1069459 T2231 C383 P25676 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1069461 T2231 C383 P25676 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1069770 T2158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1069771 T2158 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25676_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25676"}
[junit4:junit4] 2> 1069784 T2157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1069784 T2177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1069784 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1069784 T2193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1069784 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1069785 T2226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1070423 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1070424 T2135 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C385 name=collection1 org.apache.solr.core.SolrCore@4787a817 url=http://127.0.0.1:35898/collection1 node=127.0.0.1:35898_ C385_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:35898_, base_url=http://127.0.0.1:35898, leader=true}
[junit4:junit4] 2> 1070434 T2147 C385 P35898 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1070437 T2147 C385 P35898 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b52695b lockFactory=org.apache.lucene.store.NativeFSLockFactory@18c941e7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1070438 T2147 C385 P35898 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1070439 T2147 C385 P35898 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b52695b lockFactory=org.apache.lucene.store.NativeFSLockFactory@18c941e7),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b52695b lockFactory=org.apache.lucene.store.NativeFSLockFactory@18c941e7),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1070439 T2147 C385 P35898 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1070440 T2147 C385 P35898 oass.SolrIndexSearcher.<init> Opening Searcher@6cd6c1da main
[junit4:junit4] 2> 1070440 T2147 C385 P35898 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1070441 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6cd6c1da main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1070442 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> ASYNC NEW_CORE C386 name=collection1 org.apache.solr.core.SolrCore@2a5798fd url=http://127.0.0.1:34833/collection1 node=127.0.0.1:34833_ C386_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34833_, base_url=http://127.0.0.1:34833, leader=true}
[junit4:junit4] 2> 1070451 T2171 C386 P34833 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1070452 T2171 C386 P34833 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3db45d50 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f64a9fc),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1070453 T2171 C386 P34833 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1070453 T2171 C386 P34833 oass.SolrIndexSearcher.<init> Opening Searcher@71a8d430 main
[junit4:junit4] 2> 1070454 T2171 C386 P34833 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1070454 T2179 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71a8d430 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1070455 T2171 C386 P34833 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:29467/collection1/, StdNode: http://127.0.0.1:20054/collection1/, StdNode: http://127.0.0.1:25676/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1070458 T2187 C384 P20054 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1070459 T2187 C384 P20054 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@75ca288c lockFactory=org.apache.lucene.store.NativeFSLockFactory@af132b),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1070460 T2187 C384 P20054 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1070460 T2187 C384 P20054 oass.SolrIndexSearcher.<init> Opening Searcher@1e0c4612 main
[junit4:junit4] 2> 1070461 T2187 C384 P20054 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1070462 T2195 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e0c4612 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1070462 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> C383_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25676_, base_url=http://127.0.0.1:25676}
[junit4:junit4] 2> 1070464 T2216 C383 P25676 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1070466 T2216 C383 P25676 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50e1d4fa lockFactory=org.apache.lucene.store.NativeFSLockFactory@658efbd),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50e1d4fa lockFactory=org.apache.lucene.store.NativeFSLockFactory@658efbd),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1070466 T2216 C383 P25676 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1070466 T2216 C383 P25676 oass.SolrIndexSearcher.<init> Opening Searcher@23f64b71 main
[junit4:junit4] 2> 1070467 T2216 C383 P25676 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C387 name=collection1 org.apache.solr.core.SolrCore@7667c83e url=http://127.0.0.1:29467/collection1 node=127.0.0.1:29467_ C387_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:29467_, base_url=http://127.0.0.1:29467}
[junit4:junit4] 2> 1070467 T2199 C387 P29467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1070468 T2230 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23f64b71 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1070468 T2199 C387 P29467 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64da74f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d405676),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64da74f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d405676),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1070469 T2216 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1070469 T2199 C387 P29467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1070470 T2199 C387 P29467 oass.SolrIndexSearcher.<init> Opening Searcher@69df6ef7 main
[junit4:junit4] 2> 1070470 T2199 C387 P29467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1070471 T2211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69df6ef7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1070471 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 1070472 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 1070473 T2135 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1070476 T2172 C386 P34833 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1070478 T2200 C387 P29467 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1070480 T2188 C384 P20054 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1070481 T2217 C383 P25676 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1072487 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437660350440800256)} 0 2
[junit4:junit4] 2> 1072496 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1437660350444994560&update.from=http://127.0.0.1:20054/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437660350444994560)} 0 2
[junit4:junit4] 2> 1072499 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1437660350449188864&update.from=http://127.0.0.1:34833/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437660350449188864)} 0 1
[junit4:junit4] 2> 1072500 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437660350449188864)} 0 6
[junit4:junit4] 2> 1072501 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437660350444994560)} 0 12
[junit4:junit4] 2> 1072505 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1437660350459674624)]} 0 1
[junit4:junit4] 2> 1072515 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437660350467014656)]} 0 1
[junit4:junit4] 2> 1072516 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1437660350467014656)]} 0 5
[junit4:junit4] 2> 1072516 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 8
[junit4:junit4] 2> 1072520 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1437660350475403264)]} 0 1
[junit4:junit4] 2> 1072529 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437660350481694720)]} 0 1
[junit4:junit4] 2> 1072529 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1437660350481694720)]} 0 4
[junit4:junit4] 2> 1072530 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 8
[junit4:junit4] 2> 1072534 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1437660350490083328)]} 0 1
[junit4:junit4] 2> 1072543 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437660350496374784)]} 0 1
[junit4:junit4] 2> 1072543 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1437660350496374784)]} 0 4
[junit4:junit4] 2> 1072544 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 1072547 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1437660350504763392)]} 0 0
[junit4:junit4] 2> 1072554 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437660350507909120)]} 0 1
[junit4:junit4] 2> 1072554 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1437660350507909120)]} 0 4
[junit4:junit4] 2> 1072558 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1437660350515249152)]} 0 1
[junit4:junit4] 2> 1072567 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437660350521540608)]} 0 1
[junit4:junit4] 2> 1072568 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1437660350521540608)]} 0 5
[junit4:junit4] 2> 1072569 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4]} 0 9
[junit4:junit4] 2> 1072572 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1437660350529929216)]} 0 1
[junit4:junit4] 2> 1072580 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437660350536220672)]} 0 0
[junit4:junit4] 2> 1072581 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1437660350536220672)]} 0 4
[junit4:junit4] 2> 1072582 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5]} 0 7
[junit4:junit4] 2> 1072585 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1437660350544609280)]} 0 0
[junit4:junit4] 2> 1072591 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437660350547755008)]} 0 0
[junit4:junit4] 2> 1072592 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1437660350547755008)]} 0 4
[junit4:junit4] 2> 1072595 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1437660350555095040)]} 0 0
[junit4:junit4] 2> 1072602 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437660350558240768)]} 0 1
[junit4:junit4] 2> 1072602 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1437660350558240768)]} 0 4
[junit4:junit4] 2> 1072606 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1437660350565580800)]} 0 1
[junit4:junit4] 2> 1072611 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437660350568726528)]} 0 0
[junit4:junit4] 2> 1072612 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1437660350568726528)]} 0 4
[junit4:junit4] 2> 1072615 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1437660350576066560)]} 0 0
[junit4:junit4] 2> 1072621 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437660350579212288)]} 0 0
[junit4:junit4] 2> 1072622 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1437660350579212288)]} 0 4
[junit4:junit4] 2> 1072625 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1437660350586552320)]} 0 0
[junit4:junit4] 2> 1072631 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437660350589698048)]} 0 0
[junit4:junit4] 2> 1072632 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1437660350589698048)]} 0 4
[junit4:junit4] 2> 1072635 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1437660350597038080)]} 0 0
[junit4:junit4] 2> 1072643 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437660350602280960)]} 0 0
[junit4:junit4] 2> 1072644 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1437660350602280960)]} 0 4
[junit4:junit4] 2> 1072645 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11]} 0 7
[junit4:junit4] 2> 1072649 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1437660350610669568)]} 0 1
[junit4:junit4] 2> 1072655 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437660350614863872)]} 0 0
[junit4:junit4] 2> 1072656 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1437660350614863872)]} 0 4
[junit4:junit4] 2> 1072659 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1437660350622203904)]} 0 0
[junit4:junit4] 2> 1072665 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437660350625349632)]} 0 0
[junit4:junit4] 2> 1072666 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1437660350625349632)]} 0 4
[junit4:junit4] 2> 1072669 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1437660350632689664)]} 0 0
[junit4:junit4] 2> 1072675 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437660350635835392)]} 0 0
[junit4:junit4] 2> 1072676 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1437660350635835392)]} 0 4
[junit4:junit4] 2> 1072679 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1437660350643175424)]} 0 0
[junit4:junit4] 2> 1072687 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437660350648418304)]} 0 0
[junit4:junit4] 2> 1072688 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1437660350648418304)]} 0 4
[junit4:junit4] 2> 1072689 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 1072693 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1437660350656806912)]} 0 1
[junit4:junit4] 2> 1072699 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437660350661001216)]} 0 0
[junit4:junit4] 2> 1072700 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1437660350661001216)]} 0 4
[junit4:junit4] 2> 1072704 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1437660350668341248)]} 0 1
[junit4:junit4] 2> 1072710 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437660350671486976)]} 0 1
[junit4:junit4] 2> 1072710 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1437660350671486976)]} 0 4
[junit4:junit4] 2> 1072714 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1437660350678827008)]} 0 1
[junit4:junit4] 2> 1072719 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437660350681972736)]} 0 0
[junit4:junit4] 2> 1072720 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1437660350681972736)]} 0 4
[junit4:junit4] 2> 1072723 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1437660350689312768)]} 0 0
[junit4:junit4] 2> 1072731 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437660350694555648)]} 0 0
[junit4:junit4] 2> 1072732 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1437660350694555648)]} 0 4
[junit4:junit4] 2> 1072733 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19]} 0 7
[junit4:junit4] 2> 1072737 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1437660350702944256)]} 0 1
[junit4:junit4] 2> 1072747 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437660350710284288)]} 0 1
[junit4:junit4] 2> 1072747 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1437660350710284288)]} 0 4
[junit4:junit4] 2> 1072748 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20]} 0 8
[junit4:junit4] 2> 1072752 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1437660350718672896)]} 0 1
[junit4:junit4] 2> 1072759 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437660350722867200)]} 0 1
[junit4:junit4] 2> 1072760 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1437660350722867200)]} 0 5
[junit4:junit4] 2> 1072763 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1437660350730207232)]} 0 1
[junit4:junit4] 2> 1072772 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437660350736498688)]} 0 1
[junit4:junit4] 2> 1072772 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1437660350736498688)]} 0 4
[junit4:junit4] 2> 1072773 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22]} 0 8
[junit4:junit4] 2> 1072777 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1437660350744887296)]} 0 1
[junit4:junit4] 2> 1072783 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437660350748033024)]} 0 1
[junit4:junit4] 2> 1072783 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1437660350748033024)]} 0 4
[junit4:junit4] 2> 1072786 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1437660350755373056)]} 0 0
[junit4:junit4] 2> 1072793 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437660350758518784)]} 0 1
[junit4:junit4] 2> 1072793 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1437660350758518784)]} 0 4
[junit4:junit4] 2> 1072796 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1437660350765858816)]} 0 0
[junit4:junit4] 2> 1072803 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437660350769004544)]} 0 1
[junit4:junit4] 2> 1072804 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1437660350769004544)]} 0 5
[junit4:junit4] 2> 1072808 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1437660350777393152)]} 0 1
[junit4:junit4] 2> 1072816 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437660350783684608)]} 0 0
[junit4:junit4] 2> 1072817 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1437660350783684608)]} 0 4
[junit4:junit4] 2> 1072818 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26]} 0 8
[junit4:junit4] 2> 1072822 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1437660350792073216)]} 0 1
[junit4:junit4] 2> 1072832 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437660350799413248)]} 0 1
[junit4:junit4] 2> 1072832 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437660350799413248)]} 0 4
[junit4:junit4] 2> 1072833 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27]} 0 8
[junit4:junit4] 2> 1072837 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1437660350807801856)]} 0 1
[junit4:junit4] 2> 1072844 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437660350813044736)]} 0 0
[junit4:junit4] 2> 1072845 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1437660350813044736)]} 0 4
[junit4:junit4] 2> 1072846 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 1072849 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1437660350821433344)]} 0 0
[junit4:junit4] 2> 1072856 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437660350824579072)]} 0 1
[junit4:junit4] 2> 1072857 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1437660350824579072)]} 0 5
[junit4:junit4] 2> 1072860 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1437660350831919104)]} 0 1
[junit4:junit4] 2> 1072869 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437660350838210560)]} 0 1
[junit4:junit4] 2> 1072869 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1437660350838210560)]} 0 4
[junit4:junit4] 2> 1072870 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30]} 0 7
[junit4:junit4] 2> 1072873 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1437660350846599168)]} 0 0
[junit4:junit4] 2> 1072883 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437660350852890624)]} 0 1
[junit4:junit4] 2> 1072883 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437660350852890624)]} 0 4
[junit4:junit4] 2> 1072884 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31]} 0 8
[junit4:junit4] 2> 1072888 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1437660350861279232)]} 0 1
[junit4:junit4] 2> 1072895 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437660350865473536)]} 0 1
[junit4:junit4] 2> 1072895 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1437660350865473536)]} 0 4
[junit4:junit4] 2> 1072898 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1437660350872813568)]} 0 0
[junit4:junit4] 2> 1072906 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437660350878056448)]} 0 0
[junit4:junit4] 2> 1072907 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1437660350878056448)]} 0 4
[junit4:junit4] 2> 1072908 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33]} 0 7
[junit4:junit4] 2> 1072911 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1437660350886445056)]} 0 0
[junit4:junit4] 2> 1072917 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437660350889590784)]} 0 0
[junit4:junit4] 2> 1072918 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1437660350889590784)]} 0 4
[junit4:junit4] 2> 1072921 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1437660350896930816)]} 0 0
[junit4:junit4] 2> 1072929 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437660350902173696)]} 0 0
[junit4:junit4] 2> 1072930 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1437660350902173696)]} 0 4
[junit4:junit4] 2> 1072931 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35]} 0 7
[junit4:junit4] 2> 1072934 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1437660350910562304)]} 0 0
[junit4:junit4] 2> 1072940 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437660350913708032)]} 0 0
[junit4:junit4] 2> 1072941 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1437660350913708032)]} 0 4
[junit4:junit4] 2> 1072944 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1437660350921048064)]} 0 0
[junit4:junit4] 2> 1072950 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437660350924193792)]} 0 0
[junit4:junit4] 2> 1072951 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1437660350924193792)]} 0 4
[junit4:junit4] 2> 1072954 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1437660350931533824)]} 0 0
[junit4:junit4] 2> 1072961 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1437660350934679552)]} 0 1
[junit4:junit4] 2> 1072961 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1437660350934679552)]} 0 4
[junit4:junit4] 2> 1072965 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1437660350942019584)]} 0 1
[junit4:junit4] 2> 1072974 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1437660350948311040)]} 0 1
[junit4:junit4] 2> 1072974 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1437660350948311040)]} 0 4
[junit4:junit4] 2> 1072975 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39]} 0 8
[junit4:junit4] 2> 1072979 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1437660350956699648)]} 0 1
[junit4:junit4] 2> 1072986 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1437660350960893952)]} 0 0
[junit4:junit4] 2> 1072986 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1437660350960893952)]} 0 4
[junit4:junit4] 2> 1072989 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1437660350968233984)]} 0 0
[junit4:junit4] 2> 1072998 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1437660350973476864)]} 0 1
[junit4:junit4] 2> 1072998 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1437660350973476864)]} 0 4
[junit4:junit4] 2> 1072999 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41]} 0 7
[junit4:junit4] 2> 1073002 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1437660350981865472)]} 0 0
[junit4:junit4] 2> 1073009 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1437660350985011200)]} 0 1
[junit4:junit4] 2> 1073009 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1437660350985011200)]} 0 4
[junit4:junit4] 2> 1073012 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1437660350992351232)]} 0 0
[junit4:junit4] 2> 1073018 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1437660350995496960)]} 0 0
[junit4:junit4] 2> 1073019 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1437660350995496960)]} 0 4
[junit4:junit4] 2> 1073023 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1437660351002836992)]} 0 1
[junit4:junit4] 2> 1073028 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1437660351005982720)]} 0 0
[junit4:junit4] 2> 1073029 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1437660351005982720)]} 0 4
[junit4:junit4] 2> 1073032 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1437660351013322752)]} 0 0
[junit4:junit4] 2> 1073038 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1437660351016468480)]} 0 0
[junit4:junit4] 2> 1073039 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1437660351016468480)]} 0 4
[junit4:junit4] 2> 1073042 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1437660351023808512)]} 0 0
[junit4:junit4] 2> 1073049 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1437660351026954240)]} 0 1
[junit4:junit4] 2> 1073049 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1437660351026954240)]} 0 4
[junit4:junit4] 2> 1073053 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1437660351034294272)]} 0 1
[junit4:junit4] 2> 1073061 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1437660351040585728)]} 0 0
[junit4:junit4] 2> 1073062 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[47 (1437660351040585728)]} 0 4
[junit4:junit4] 2> 1073063 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47]} 0 7
[junit4:junit4] 2> 1073066 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1437660351048974336)]} 0 0
[junit4:junit4] 2> 1073075 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1437660351054217216)]} 0 1
[junit4:junit4] 2> 1073075 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1437660351054217216)]} 0 4
[junit4:junit4] 2> 1073076 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48]} 0 7
[junit4:junit4] 2> 1073079 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1437660351062605824)]} 0 0
[junit4:junit4] 2> 1073087 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1437660351067848704)]} 0 0
[junit4:junit4] 2> 1073088 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[49 (1437660351067848704)]} 0 4
[junit4:junit4] 2> 1073089 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49]} 0 7
[junit4:junit4] 2> 1073094 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1437660351077285888)]} 0 1
[junit4:junit4] 2> 1073104 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1437660351084625920)]} 0 1
[junit4:junit4] 2> 1073105 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1437660351084625920)]} 0 5
[junit4:junit4] 2> 1073105 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50]} 0 8
[junit4:junit4] 2> 1073109 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1437660351093014528)]} 0 1
[junit4:junit4] 2> 1073117 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1437660351099305984)]} 0 0
[junit4:junit4] 2> 1073118 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1437660351099305984)]} 0 4
[junit4:junit4] 2> 1073119 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51]} 0 8
[junit4:junit4] 2> 1073121 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1437660351106646016)]} 0 0
[junit4:junit4] 2> 1073127 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1437660351109791744)]} 0 0
[junit4:junit4] 2> 1073128 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1437660351109791744)]} 0 4
[junit4:junit4] 2> 1073131 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1437660351117131776)]} 0 0
[junit4:junit4] 2> 1073139 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1437660351122374656)]} 0 0
[junit4:junit4] 2> 1073140 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[53 (1437660351122374656)]} 0 4
[junit4:junit4] 2> 1073140 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53]} 0 6
[junit4:junit4] 2> 1073143 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1437660351129714688)]} 0 0
[junit4:junit4] 2> 1073151 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1437660351134957568)]} 0 0
[junit4:junit4] 2> 1073152 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1437660351134957568)]} 0 4
[junit4:junit4] 2> 1073152 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54]} 0 6
[junit4:junit4] 2> 1073155 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1437660351142297600)]} 0 0
[junit4:junit4] 2> 1073163 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1437660351147540480)]} 0 0
[junit4:junit4] 2> 1073164 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1437660351147540480)]} 0 4
[junit4:junit4] 2> 1073164 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55]} 0 6
[junit4:junit4] 2> 1073167 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1437660351154880512)]} 0 0
[junit4:junit4] 2> 1073174 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1437660351158026240)]} 0 1
[junit4:junit4] 2> 1073174 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1437660351158026240)]} 0 4
[junit4:junit4] 2> 1073177 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1437660351165366272)]} 0 0
[junit4:junit4] 2> 1073186 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1437660351171657728)]} 0 0
[junit4:junit4] 2> 1073187 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1437660351171657728)]} 0 4
[junit4:junit4] 2> 1073188 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57]} 0 8
[junit4:junit4] 2> 1073190 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1437660351178997760)]} 0 0
[junit4:junit4] 2> 1073196 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1437660351182143488)]} 0 0
[junit4:junit4] 2> 1073197 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1437660351182143488)]} 0 4
[junit4:junit4] 2> 1073200 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1437660351189483520)]} 0 0
[junit4:junit4] 2> 1073208 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1437660351194726400)]} 0 0
[junit4:junit4] 2> 1073209 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1437660351194726400)]} 0 4
[junit4:junit4] 2> 1073210 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59]} 0 7
[junit4:junit4] 2> 1073213 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1437660351202066432)]} 0 1
[junit4:junit4] 2> 1073222 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1437660351208357888)]} 0 1
[junit4:junit4] 2> 1073222 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1437660351208357888)]} 0 4
[junit4:junit4] 2> 1073223 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60]} 0 7
[junit4:junit4] 2> 1073226 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1437660351216746496)]} 0 0
[junit4:junit4] 2> 1073232 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1437660351219892224)]} 0 0
[junit4:junit4] 2> 1073233 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1437660351219892224)]} 0 4
[junit4:junit4] 2> 1073236 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1437660351227232256)]} 0 0
[junit4:junit4] 2> 1073243 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1437660351230377984)]} 0 1
[junit4:junit4] 2> 1073243 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1437660351230377984)]} 0 4
[junit4:junit4] 2> 1073246 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1437660351237718016)]} 0 0
[junit4:junit4] 2> 1073252 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1437660351240863744)]} 0 0
[junit4:junit4] 2> 1073253 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1437660351240863744)]} 0 4
[junit4:junit4] 2> 1073256 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1437660351248203776)]} 0 0
[junit4:junit4] 2> 1073262 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1437660351251349504)]} 0 0
[junit4:junit4] 2> 1073263 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1437660351251349504)]} 0 4
[junit4:junit4] 2> 1073265 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1437660351257640960)]} 0 0
[junit4:junit4] 2> 1073274 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1437660351262883840)]} 0 1
[junit4:junit4] 2> 1073274 T2173 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[65 (1437660351262883840)]} 0 4
[junit4:junit4] 2> 1073275 T2189 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65]} 0 7
[junit4:junit4] 2> 1073278 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1437660351271272448)]} 0 0
[junit4:junit4] 2> 1073284 T2199 C387 P29467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[66 (1437660351274418176)]} 0 0
[junit4:junit4] 2> 1073285 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1437660351274418176)]} 0 4
[junit4:junit4] 2> 1073288 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1437660351281758208)]} 0 0
[junit4:junit4] 2> 1073297 T2218 C383 P25676 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20054/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[67 (1437660351287001088)]} 0 1
[junit4:junit4] 2> 1073297 T2187 C384 P20054 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34833/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[67 (1437660351287001088)]} 0 4
[junit4:junit4] 2> 1073298 T2171 C386 P34833 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67]} 0 7
[junit4:junit4] 2> 1073302 T2147 C385 P35898 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1437660351295389696)]} 0 1
[junit4:junit4]
[...truncated too long message...]
lection1_shard1_1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1_1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":null,
[junit4:junit4] 2> "core":"collection1_shard1_1_replica1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29467"}
[junit4:junit4] 2> 1250440 T2294 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1250445 T2135 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 25676
[junit4:junit4] 2> 1250445 T2135 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1390355055
[junit4:junit4] 2> 1251448 T2135 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1251448 T2135 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1251453 T2135 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@e72490d
[junit4:junit4] 2> 1251455 T2135 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=792,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1251455 T2135 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1251456 T2135 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1251456 T2135 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1251459 T2135 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1251459 T2135 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1251460 T2135 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index;done=false>>]
[junit4:junit4] 2> 1251460 T2135 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4/index
[junit4:junit4] 2> 1251461 T2135 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4;done=false>>]
[junit4:junit4] 2> 1251461 T2135 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865/jetty4
[junit4:junit4] 2> 1251462 T2294 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89853770063740940-127.0.0.1:25676_-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 1251464 T2135 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 1251535 T2135 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1251537 T2135 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:63758 63758
[junit4:junit4] 2> 1251692 T2209 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1251793 T2209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1251794 T2209 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1251795 T2135 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 35898
[junit4:junit4] 2> !!!! WARNING: best effort to remove /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371059723865 FAILED !!!!!
[junit4:junit4] 2> 1251796 T2135 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1251796 T2135 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:63758 63758
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=D53082544D8F9C02 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_PH -Dtests.timezone=America/Barbados -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 208s J0 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:34833 returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D53082544D8F9C02:54D60C4C3AD0FC3E]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1251943 T2135 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 207535 T2134 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 1252764 T2226 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1252764 T2226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1252764 T2226 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {id=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=MockSep), n_ti=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_PH, timezone=America/Barbados
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=230983640,total=521142272
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestCopyFieldCollectionResource, SampleTest, DirectUpdateHandlerOptimizeTest, TestReloadAndDeleteDocs, TestLuceneMatchVersion, TermVectorComponentDistributedTest, TestPerFieldSimilarity, TestSolrXml, FieldAnalysisRequestHandlerTest, TestTrie, TestStressVersions, TestFuzzyAnalyzedSuggestions, TestFiltering, AliasIntegrationTest, TestSuggestSpellingConverter, DistanceFunctionTest, TestFoldingMultitermQuery, RAMDirectoryFactoryTest, MultiTermTest, TestPhraseSuggestions, TestSchemaResource, TestDynamicFieldCollectionResource, TestCloudManagedSchemaAddField, DocumentBuilderTest, TestJoin, TimeZoneUtilsTest, DistributedTermsComponentTest, TestJmxIntegration, TestMergePolicyConfig, TestLRUCache, TestBM25SimilarityFactory, PreAnalyzedUpdateProcessorTest, UUIDFieldTest, TestAtomicUpdateErrorCases, TestWriterPerf, SignatureUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, JsonLoaderTest, BasicFunctionalityTest, SpellCheckCollatorTest, BasicDistributedZk2Test, JSONWriterTest, SolrCoreCheckLockOnStartupTest, TestBinaryField, TestSolrDeletionPolicy2, TestPropInject, TestCharFilters, CachingDirectoryFactoryTest, PrimUtilsTest, SliceStateUpdateTest, ShardRoutingCustomTest, TestElisionMultitermQuery, TestZkChroot, TestLMDirichletSimilarityFactory, TestFastWriter, TestLMJelinekMercerSimilarityFactory, DateMathParserTest, TestHashPartitioner, CopyFieldTest, TestArbitraryIndexDir, OverseerTest, TestQueryUtils, TestAnalyzedSuggestions, DebugComponentTest, SolrCmdDistributorTest, CircularListTest, IndexBasedSpellCheckerTest, BadCopyFieldTest, TestDFRSimilarityFactory, TestFieldResource, TestSort, TestReload, BasicZkTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 208.58s, 1 test, 1 error <<< FAILURES!
[...truncated 463 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 1 error, 13 ignored (7 assumptions)
Total time: 52 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1328 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1328/
1 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([94C1FB14CE220E9A:1527750CB97D6EA6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9387 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 539211 T860 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /y_q/
[junit4:junit4] 2> 539218 T860 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1371141565911
[junit4:junit4] 2> 539220 T860 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 539220 T861 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 539321 T860 oasc.ZkTestServer.run start zk server on port:55935
[junit4:junit4] 2> 539322 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 539326 T867 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1426241c name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 539327 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 539327 T860 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 539341 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 539342 T869 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@605d5c9c name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 539343 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 539343 T860 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 539352 T860 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 539355 T860 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 539358 T860 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 539367 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 539368 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 539380 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 539381 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 539485 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 539486 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 539495 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 539496 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 539499 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 539499 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 539503 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 539503 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 539507 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 539507 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 539511 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 539511 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 539514 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 539515 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 539771 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 539775 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:27612
[junit4:junit4] 2> 539776 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 539777 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 539777 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211
[junit4:junit4] 2> 539792 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/solr.xml
[junit4:junit4] 2> 539793 T860 oasc.CoreContainer.<init> New CoreContainer 1156464092
[junit4:junit4] 2> 539794 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/'
[junit4:junit4] 2> 539794 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/'
[junit4:junit4] 2> 539924 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 539925 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 539926 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 539927 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 539927 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 539928 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 539928 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 539929 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 539929 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 539930 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 539949 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 539950 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4] 2> 539951 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 539952 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 539955 T880 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5005e58 name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 539956 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 539959 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 539972 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 539975 T882 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43256374 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 539975 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 539977 T860 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 539987 T860 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 539990 T860 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 539997 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27612_y_q
[junit4:junit4] 2> 539999 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27612_y_q
[junit4:junit4] 2> 540002 T860 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 540018 T860 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 540026 T860 oasc.Overseer.start Overseer (id=89859133664067587-127.0.0.1:27612_y_q-n_0000000000) starting
[junit4:junit4] 2> 540036 T860 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 540046 T884 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 540047 T860 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 540049 T860 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 540057 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 540059 T883 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 540064 T885 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 540064 T885 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 541564 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 541565 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:27612_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:27612/y_q"}
[junit4:junit4] 2> 541565 T883 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 541565 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 541611 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 542068 T885 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1
[junit4:junit4] 2> 542068 T885 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 542069 T885 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 542069 T885 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 542071 T885 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/'
[junit4:junit4] 2> 542072 T885 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/lib/README' to classloader
[junit4:junit4] 2> 542073 T885 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 542127 T885 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 542189 T885 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 542291 T885 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 542297 T885 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 542871 T885 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 542875 T885 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 542878 T885 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 542883 T885 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 542913 T885 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 542914 T885 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/
[junit4:junit4] 2> 542914 T885 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4] 2> 542915 T885 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 542915 T885 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data
[junit4:junit4] 2> 542916 T885 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index/
[junit4:junit4] 2> 542916 T885 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 542917 T885 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index
[junit4:junit4] 2> 542922 T885 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4] 2> 542922 T885 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 542925 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 542926 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 542927 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 542927 T885 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 542928 T885 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 542928 T885 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 542929 T885 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 542930 T885 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 542930 T885 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 542946 T885 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 542954 T885 oass.SolrIndexSearcher.<init> Opening Searcher@1b263d21 main
[junit4:junit4] 2> 542955 T885 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/tlog
[junit4:junit4] 2> 542956 T885 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 542956 T885 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 542960 T886 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b263d21 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 542963 T885 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 542963 T885 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:27612/y_q collection:control_collection shard:shard1
[junit4:junit4] 2> 542964 T885 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 542981 T885 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 542991 T885 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 542991 T885 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 542992 T885 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4] 2> 542992 T885 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 542992 T885 oasc.SyncStrategy.syncToMe http://127.0.0.1:27612/y_q/collection1/ has no replicas
[junit4:junit4] 2> 542993 T885 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4] 2> 542993 T885 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 543115 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 543155 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 543212 T885 oasc.ZkController.register We are http://127.0.0.1:27612/y_q/collection1/ and leader is http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4] 2> 543212 T885 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:27612/y_q
[junit4:junit4] 2> 543212 T885 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 543213 T885 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 543213 T885 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 543215 T885 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 543216 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 543217 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 543218 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 543236 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 543238 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 543240 T889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ec22862 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 543241 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 543243 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 543245 T860 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 543507 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 543511 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41428
[junit4:junit4] 2> 543512 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 543513 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 543513 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938
[junit4:junit4] 2> 543514 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/solr.xml
[junit4:junit4] 2> 543514 T860 oasc.CoreContainer.<init> New CoreContainer 1618698809
[junit4:junit4] 2> 543515 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/'
[junit4:junit4] 2> 543516 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/'
[junit4:junit4] 2> 543645 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 543646 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 543646 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 543647 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 543648 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 543648 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 543649 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 543649 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 543650 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 543651 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 543669 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 543670 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4] 2> 543670 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 543672 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 543675 T900 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6edb135a name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 543676 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 543691 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 543704 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 543706 T902 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72f46074 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 543707 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 543716 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 544666 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 544667 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:27612_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:27612/y_q"}
[junit4:junit4] 2> 544682 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544682 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544682 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544720 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41428_y_q
[junit4:junit4] 2> 544722 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41428_y_q
[junit4:junit4] 2> 544724 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544724 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544724 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 544726 T882 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 544733 T902 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 544733 T889 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 544738 T903 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 544738 T903 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 546187 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 546188 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41428_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41428/y_q"}
[junit4:junit4] 2> 546189 T883 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 546189 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 546209 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 546209 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 546209 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 546742 T903 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1
[junit4:junit4] 2> 546742 T903 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 546743 T903 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 546743 T903 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 546745 T903 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/'
[junit4:junit4] 2> 546747 T903 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 546747 T903 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/lib/README' to classloader
[junit4:junit4] 2> 546802 T903 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 546863 T903 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 546965 T903 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 546971 T903 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 547546 T903 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 547551 T903 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 547553 T903 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 547559 T903 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 547589 T903 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 547589 T903 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/
[junit4:junit4] 2> 547590 T903 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4] 2> 547591 T903 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 547591 T903 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1
[junit4:junit4] 2> 547592 T903 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index/
[junit4:junit4] 2> 547592 T903 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 547593 T903 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index
[junit4:junit4] 2> 547597 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 547597 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 547601 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 547601 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 547602 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 547603 T903 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 547604 T903 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 547604 T903 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 547605 T903 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 547605 T903 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 547606 T903 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 547622 T903 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 547629 T903 oass.SolrIndexSearcher.<init> Opening Searcher@7bd7d288 main
[junit4:junit4] 2> 547629 T903 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/tlog
[junit4:junit4] 2> 547630 T903 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 547631 T903 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 547635 T904 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7bd7d288 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 547638 T903 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 547638 T903 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41428/y_q collection:collection1 shard:shard1
[junit4:junit4] 2> 547639 T903 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 547662 T903 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 547673 T903 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 547673 T903 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 547674 T903 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4] 2> 547674 T903 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 547674 T903 oasc.SyncStrategy.syncToMe http://127.0.0.1:41428/y_q/collection1/ has no replicas
[junit4:junit4] 2> 547675 T903 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4] 2> 547675 T903 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 547713 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 547729 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 547729 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 547729 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 547732 T903 oasc.ZkController.register We are http://127.0.0.1:41428/y_q/collection1/ and leader is http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4] 2> 547732 T903 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41428/y_q
[junit4:junit4] 2> 547732 T903 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 547733 T903 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 547733 T903 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 547734 T903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 547735 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 547736 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 547737 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 548001 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 548004 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14096
[junit4:junit4] 2> 548005 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 548005 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 548006 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446
[junit4:junit4] 2> 548006 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/solr.xml
[junit4:junit4] 2> 548007 T860 oasc.CoreContainer.<init> New CoreContainer 1138398660
[junit4:junit4] 2> 548008 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/'
[junit4:junit4] 2> 548008 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/'
[junit4:junit4] 2> 548128 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 548129 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 548130 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 548130 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 548131 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 548132 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 548132 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 548133 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 548133 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 548134 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 548151 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 548152 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4] 2> 548152 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 548153 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 548157 T916 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@681892be name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 548157 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 548172 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 548186 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 548188 T918 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab921e9 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 548188 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 548194 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 549198 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14096_y_q
[junit4:junit4] 2> 549200 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14096_y_q
[junit4:junit4] 2> 549203 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549203 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549203 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549204 T918 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 549214 T882 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 549214 T902 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 549214 T889 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 549219 T919 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 549220 T919 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 549234 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 549235 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41428_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41428/y_q"}
[junit4:junit4] 2> 549239 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:14096_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:14096/y_q"}
[junit4:junit4] 2> 549240 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 549240 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 549251 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 549251 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 549251 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 549251 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 550222 T919 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1
[junit4:junit4] 2> 550222 T919 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 550223 T919 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 550224 T919 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 550225 T919 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/'
[junit4:junit4] 2> 550227 T919 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/lib/README' to classloader
[junit4:junit4] 2> 550228 T919 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 550282 T919 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 550343 T919 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 550445 T919 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 550451 T919 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 551029 T919 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 551034 T919 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 551036 T919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 551041 T919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 551071 T919 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 551072 T919 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/
[junit4:junit4] 2> 551072 T919 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4] 2> 551073 T919 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 551074 T919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2
[junit4:junit4] 2> 551074 T919 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index/
[junit4:junit4] 2> 551075 T919 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 551075 T919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index
[junit4:junit4] 2> 551079 T919 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 551080 T919 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 551084 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 551084 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 551085 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 551086 T919 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 551086 T919 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 551087 T919 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 551087 T919 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 551088 T919 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 551089 T919 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 551105 T919 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 551113 T919 oass.SolrIndexSearcher.<init> Opening Searcher@3b20df58 main
[junit4:junit4] 2> 551114 T919 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/tlog
[junit4:junit4] 2> 551115 T919 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 551115 T919 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 551121 T920 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b20df58 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 551123 T919 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 551124 T919 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14096/y_q collection:collection1 shard:shard2
[junit4:junit4] 2> 551125 T919 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 551134 T919 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 551151 T919 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 551151 T919 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 551151 T919 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4] 2> 551152 T919 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 551152 T919 oasc.SyncStrategy.syncToMe http://127.0.0.1:14096/y_q/collection1/ has no replicas
[junit4:junit4] 2> 551153 T919 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4] 2> 551153 T919 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 552267 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 552292 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552292 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552292 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552292 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552331 T919 oasc.ZkController.register We are http://127.0.0.1:14096/y_q/collection1/ and leader is http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4] 2> 552331 T919 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14096/y_q
[junit4:junit4] 2> 552331 T919 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 552332 T919 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 552332 T919 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 552334 T919 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 552335 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 552336 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 552337 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 552622 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 552626 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62951
[junit4:junit4] 2> 552627 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 552628 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 552628 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047
[junit4:junit4] 2> 552629 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/solr.xml
[junit4:junit4] 2> 552629 T860 oasc.CoreContainer.<init> New CoreContainer 173336716
[junit4:junit4] 2> 552630 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/'
[junit4:junit4] 2> 552631 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/'
[junit4:junit4] 2> 552761 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 552762 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 552762 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 552763 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 552764 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 552764 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 552765 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 552765 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 552766 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 552767 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 552785 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 552786 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4] 2> 552787 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 552788 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 552791 T932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e8d50cd name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 552792 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 552808 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 552822 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 552825 T934 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34a846c9 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 552825 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 552834 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 553805 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 553806 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:14096_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:14096/y_q"}
[junit4:junit4] 2> 553822 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553822 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553822 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553822 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553822 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553838 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62951_y_q
[junit4:junit4] 2> 553840 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62951_y_q
[junit4:junit4] 2> 553843 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553843 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553843 T918 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 553843 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553843 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 553843 T934 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 553844 T882 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 553844 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 553850 T902 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 553851 T889 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 553856 T935 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 553856 T935 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 555327 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 555345 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555345 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555345 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555345 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555345 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555860 T935 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1
[junit4:junit4] 2> 555860 T935 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 555861 T935 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 555862 T935 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 555863 T935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/'
[junit4:junit4] 2> 555865 T935 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/lib/README' to classloader
[junit4:junit4] 2> 555866 T935 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 555920 T935 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 555982 T935 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 556084 T935 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 556090 T935 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 556645 T935 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 556648 T935 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 556650 T935 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 556654 T935 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 556677 T935 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 556678 T935 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/
[junit4:junit4] 2> 556678 T935 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4] 2> 556678 T935 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 556679 T935 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3
[junit4:junit4] 2> 556679 T935 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index/
[junit4:junit4] 2> 556679 T935 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 556680 T935 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index
[junit4:junit4] 2> 556683 T935 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_1,generation=1}
[junit4:junit4] 2> 556683 T935 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 556686 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 556686 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 556687 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 556687 T935 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 556689 T935 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 556689 T935 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 556700 T935 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 556705 T935 oass.SolrIndexSearcher.<init> Opening Searcher@6ba1d1c8 main
[junit4:junit4] 2> 556705 T935 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/tlog
[junit4:junit4] 2> 556706 T935 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 556706 T935 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 556710 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ba1d1c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 556713 T935 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 556713 T935 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62951/y_q collection:collection1 shard:shard1
[junit4:junit4] 2> 556716 T935 oasc.ZkController.register We are http://127.0.0.1:62951/y_q/collection1/ and leader is http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4] 2> 556716 T935 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62951/y_q
[junit4:junit4] 2> 556716 T935 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 556717 T935 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C157 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C157_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4] 2> 556717 T937 C157 P62951 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 556717 T935 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 556718 T937 C157 P62951 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 556718 T937 C157 P62951 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 556719 T937 C157 P62951 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 556719 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 556720 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 556720 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 556721 T937 C157 P62951 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 556743 T892 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 556849 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 556850 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 556868 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 556868 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 556868 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 556868 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 556868 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557008 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 557011 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32634
[junit4:junit4] 2> 557012 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 557012 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 557013 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434
[junit4:junit4] 2> 557014 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/solr.xml
[junit4:junit4] 2> 557014 T860 oasc.CoreContainer.<init> New CoreContainer 1242169106
[junit4:junit4] 2> 557015 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/'
[junit4:junit4] 2> 557015 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/'
[junit4:junit4] 2> 557145 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 557145 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 557146 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 557147 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 557147 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 557148 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 557148 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 557149 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 557149 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 557150 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 557169 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 557169 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4] 2> 557170 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 557171 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 557174 T949 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c443ec6 name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 557175 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 557177 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 557186 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 557188 T951 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19c79ded name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 557189 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 557201 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 557745 T892 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 557745 T892 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:62951_y_q&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 558204 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32634_y_q
[junit4:junit4] 2> 558210 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32634_y_q
[junit4:junit4] 2> 558213 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 558214 T934 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558213 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 558213 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 558214 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558214 T918 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558214 T951 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558216 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558215 T882 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558227 T902 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558227 T889 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 558232 T952 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 558232 T952 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 558372 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 558373 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4] 2> 558374 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 558374 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 558388 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558388 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558388 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558388 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558388 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558388 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559235 T952 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1
[junit4:junit4] 2> 559235 T952 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 559236 T952 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 559236 T952 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 559238 T952 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/'
[junit4:junit4] 2> 559239 T952 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/lib/README' to classloader
[junit4:junit4] 2> 559240 T952 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 559293 T952 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 559367 T952 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 559469 T952 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 559475 T952 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C158 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C158_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4] 2> 559747 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41428/y_q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 559747 T937 C158 P62951 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62951/y_q START replicas=[http://127.0.0.1:41428/y_q/collection1/] nUpdates=100
[junit4:junit4] 2> 559748 T937 C158 P62951 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 559750 T937 C158 P62951 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 559750 T937 C158 P62951 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41428/y_q/collection1/. core=collection1
[junit4:junit4] 2> 559750 T937 C158 P62951 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C159 name=collection1 org.apache.solr.core.SolrCore@61c2d960 url=http://127.0.0.1:41428/y_q/collection1 node=127.0.0.1:41428_y_q C159_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41428_y_q, base_url=http://127.0.0.1:41428/y_q, leader=true}
[junit4:junit4] 2> 559757 T893 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 559764 T894 C159 P41428 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 559767 T894 C159 P41428 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 559767 T894 C159 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 559768 T894 C159 P41428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_2,generation=2}
[junit4:junit4] 2> 559769 T894 C159 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 559770 T894 C159 P41428 oass.SolrIndexSearcher.<init> Opening Searcher@3d4ef04f realtime
[junit4:junit4] 2> 559770 T894 C159 P41428 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 559771 T894 C159 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 559772 T937 C158 P62951 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 559772 T937 C158 P62951 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 559774 T895 C159 P41428 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 559775 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 559776 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 559776 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 559777 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 559779 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 559780 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 559781 T937 C158 P62951 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4] 2> 559782 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@11697660 lockFactory=org.apache.lucene.store.NativeFSLockFactory@49b5ec72) fullCopy=false
[junit4:junit4] 2> 559784 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 559785 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 559786 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 559787 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 559789 T937 C158 P62951 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_2,generation=2}
[junit4:junit4] 2> 559790 T937 C158 P62951 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 559790 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 559791 T937 C158 P62951 oass.SolrIndexSearcher.<init> Opening Searcher@3e84a226 main
[junit4:junit4] 2> 559792 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e84a226 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 559793 T937 C158 P62951 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473;done=true>>]
[junit4:junit4] 2> 559793 T937 C158 P62951 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4] 2> 559794 T937 C158 P62951 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4] 2> 559794 T937 C158 P62951 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 559794 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 559794 T937 C158 P62951 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 559795 T937 C158 P62951 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 559797 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 559894 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 559895 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 559908 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559909 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559908 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559908 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559908 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 559908 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560036 T952 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 560040 T952 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 560042 T952 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 560048 T952 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 560075 T952 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 560075 T952 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/
[junit4:junit4] 2> 560076 T952 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4] 2> 560076 T952 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 560077 T952 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4
[junit4:junit4] 2> 560078 T952 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index/
[junit4:junit4] 2> 560078 T952 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 560079 T952 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index
[junit4:junit4] 2> 560083 T952 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 560083 T952 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 560087 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 560087 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 560088 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 560089 T952 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 560090 T952 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 560090 T952 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 560091 T952 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 560092 T952 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 560092 T952 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 560109 T952 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 560116 T952 oass.SolrIndexSearcher.<init> Opening Searcher@b68096e main
[junit4:junit4] 2> 560117 T952 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/tlog
[junit4:junit4] 2> 560118 T952 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 560119 T952 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 560124 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b68096e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 560126 T952 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 560127 T952 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32634/y_q collection:collection1 shard:shard2
[junit4:junit4] 2> 560130 T952 oasc.ZkController.register We are http://127.0.0.1:32634/y_q/collection1/ and leader is http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4] 2> 560130 T952 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32634/y_q
[junit4:junit4] 2> 560131 T952 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 560131 T952 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C160 name=collection1 org.apache.solr.core.SolrCore@5b967893 url=http://127.0.0.1:32634/y_q/collection1 node=127.0.0.1:32634_y_q C160_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4] 2> 560132 T956 C160 P32634 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 560132 T952 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560132 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 560133 T956 C160 P32634 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 560133 T956 C160 P32634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 560133 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 560134 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 560135 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 560135 T956 C160 P32634 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 560160 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560161 T908 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 560161 T860 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 560162 T860 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 560163 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 561165 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 561413 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 561414 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4] 2> 561436 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561436 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561436 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561436 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561436 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561436 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 562163 T908 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 562163 T908 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:32634_y_q&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 562168 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 563170 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C160_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4] 2> 564165 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:14096/y_q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 564165 T956 C160 P32634 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32634/y_q START replicas=[http://127.0.0.1:14096/y_q/collection1/] nUpdates=100
[junit4:junit4] 2> 564166 T956 C160 P32634 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 564167 T956 C160 P32634 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 564168 T956 C160 P32634 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:14096/y_q/collection1/. core=collection1
[junit4:junit4] 2> 564168 T956 C160 P32634 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 564172 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C161 name=collection1 org.apache.solr.core.SolrCore@56edda62 url=http://127.0.0.1:14096/y_q/collection1 node=127.0.0.1:14096_y_q C161_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:14096_y_q, base_url=http://127.0.0.1:14096/y_q, leader=true}
[junit4:junit4] 2> 564175 T909 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 564181 T910 C161 P14096 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 564184 T910 C161 P14096 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 564185 T910 C161 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 564186 T910 C161 P14096 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 564186 T910 C161 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 564187 T910 C161 P14096 oass.SolrIndexSearcher.<init> Opening Searcher@3eecc8ce realtime
[junit4:junit4] 2> 564187 T910 C161 P14096 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564188 T910 C161 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 564189 T956 C160 P32634 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 564189 T956 C160 P32634 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 564191 T911 C161 P14096 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 564191 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 564192 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 564192 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 564193 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 564195 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 564196 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 564197 T956 C160 P32634 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4] 2> 564198 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@322786b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41246224) fullCopy=false
[junit4:junit4] 2> 564201 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 564203 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 564204 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 564204 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 564206 T956 C160 P32634 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_2,generation=2}
[junit4:junit4] 2> 564207 T956 C160 P32634 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 564207 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 564208 T956 C160 P32634 oass.SolrIndexSearcher.<init> Opening Searcher@1b83f973 main
[junit4:junit4] 2> 564209 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b83f973 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889;done=true>>]
[junit4:junit4] 2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4] 2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4] 2> 564211 T956 C160 P32634 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 564211 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 564211 T956 C160 P32634 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 564212 T956 C160 P32634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 564213 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 564442 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 564443 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4] 2> 564462 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564462 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564462 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564462 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564462 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564462 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 565174 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 565175 T860 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C162 name=collection1 org.apache.solr.core.SolrCore@7a189a1 url=http://127.0.0.1:27612/y_q/collection1 node=127.0.0.1:27612_y_q C162_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:27612_y_q, base_url=http://127.0.0.1:27612/y_q, leader=true}
[junit4:junit4] 2> 565184 T872 C162 P27612 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 565188 T872 C162 P27612 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4] 2> 565189 T872 C162 P27612 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 565190 T872 C162 P27612 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_2,generation=2}
[junit4:junit4] 2> 565190 T872 C162 P27612 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 565191 T872 C162 P27612 oass.SolrIndexSearcher.<init> Opening Searcher@c73606 main
[junit4:junit4] 2> 565192 T872 C162 P27612 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 565193 T886 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c73606 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 565193 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> ASYNC NEW_CORE C163 name=collection1 org.apache.solr.core.SolrCore@56edda62 url=http://127.0.0.1:14096/y_q/collection1 node=127.0.0.1:14096_y_q C163_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:14096_y_q, base_url=http://127.0.0.1:14096/y_q, leader=true}
[junit4:junit4] 2> 565206 T912 C163 P14096 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 565207 T912 C163 P14096 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 565208 T912 C163 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 565209 T912 C163 P14096 oass.SolrIndexSearcher.<init> Opening Searcher@13479058 main
[junit4:junit4] 2> 565209 T912 C163 P14096 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 565210 T920 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13479058 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 565211 T912 C163 P14096 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41428/y_q/collection1/, StdNode: http://127.0.0.1:62951/y_q/collection1/, StdNode: http://127.0.0.1:32634/y_q/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C164 name=collection1 org.apache.solr.core.SolrCore@61c2d960 url=http://127.0.0.1:41428/y_q/collection1 node=127.0.0.1:41428_y_q C164_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41428_y_q, base_url=http://127.0.0.1:41428/y_q, leader=true}
[junit4:junit4] 2> 565214 T896 C164 P41428 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 565217 T896 C164 P41428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_3,generation=3}
[junit4:junit4] 2> 565217 T896 C164 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 565218 T896 C164 P41428 oass.SolrIndexSearcher.<init> Opening Searcher@60c3fd55 main
[junit4:junit4] 2> 565218 T896 C164 P41428 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 565219 T904 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60c3fd55 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 565220 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C165 name=collection1 org.apache.solr.core.SolrCore@5b967893 url=http://127.0.0.1:32634/y_q/collection1 node=127.0.0.1:32634_y_q C165_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4] 2> 565223 T941 C165 P32634 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C166 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C166_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4] 2> 565223 T924 C166 P62951 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 565224 T941 C165 P32634 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_3,generation=3}
[junit4:junit4] 2> 565224 T924 C166 P62951 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_3,generation=3}
[junit4:junit4] 2> 565225 T941 C165 P32634 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 565225 T924 C166 P62951 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 565226 T941 C165 P32634 oass.SolrIndexSearcher.<init> Opening Searcher@16386e42 main
[junit4:junit4] 2> 565226 T924 C166 P62951 oass.SolrIndexSearcher.<init> Opening Searcher@4df1c78c main
[junit4:junit4] 2> 565227 T941 C165 P32634 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 565227 T924 C166 P62951 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 565228 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16386e42 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 565228 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4df1c78c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 565229 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 565229 T924 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 565230 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 565232 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 565235 T897 C164 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 565237 T925 C166 P62951 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 565239 T913 C163 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 565242 T942 C165 P32634 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 567247 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437746168005132288)} 0 2
[junit4:junit4] 2> 567257 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=FROMLEADER&_version_=-1437746168009326592&update.from=http://127.0.0.1:41428/y_q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168009326592)} 0 3
[junit4:junit4] 2> 567261 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=FROMLEADER&_version_=-1437746168014569472&update.from=http://127.0.0.1:14096/y_q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168014569472)} 0 2
[junit4:junit4] 2> 567261 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168014569472)} 0 7
[junit4:junit4] 2> 567262 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437746168009326592)} 0 12
[junit4:junit4] 2> 567267 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[0 (1437746168025055232)]} 0 2
[junit4:junit4] 2> 567274 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437746168030298112)]} 0 1
[junit4:junit4] 2> 567275 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[0 (1437746168030298112)]} 0 5
[junit4:junit4] 2> 567279 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[1 (1437746168038686720)]} 0 1
[junit4:junit4] 2> 567286 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437746168042881024)]} 0 1
[junit4:junit4] 2> 567286 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[1 (1437746168042881024)]} 0 4
[junit4:junit4] 2> 567290 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[2 (1437746168050221056)]} 0 1
[junit4:junit4] 2> 567299 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437746168056512512)]} 0 1
[junit4:junit4] 2> 567301 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1437746168056512512)]} 0 5
[junit4:junit4] 2> 567301 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[2]} 0 9
[junit4:junit4] 2> 567305 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[3 (1437746168065949696)]} 0 1
[junit4:junit4] 2> 567314 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437746168072241152)]} 0 1
[junit4:junit4] 2> 567314 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1437746168072241152)]} 0 4
[junit4:junit4] 2> 567315 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4] 2> 567319 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[4 (1437746168080629760)]} 0 1
[junit4:junit4] 2> 567326 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437746168084824064)]} 0 1
[junit4:junit4] 2> 567326 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[4 (1437746168084824064)]} 0 4
[junit4:junit4] 2> 567330 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[5 (1437746168092164096)]} 0 1
[junit4:junit4] 2> 567336 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437746168095309824)]} 0 1
[junit4:junit4] 2> 567336 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[5 (1437746168095309824)]} 0 4
[junit4:junit4] 2> 567340 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[6 (1437746168102649856)]} 0 1
[junit4:junit4] 2> 567349 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437746168108941312)]} 0 1
[junit4:junit4] 2> 567349 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1437746168108941312)]} 0 4
[junit4:junit4] 2> 567350 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[6]} 0 7
[junit4:junit4] 2> 567354 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[7 (1437746168117329920)]} 0 1
[junit4:junit4] 2> 567361 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437746168121524224)]} 0 1
[junit4:junit4] 2> 567361 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[7 (1437746168121524224)]} 0 4
[junit4:junit4] 2> 567365 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[8 (1437746168128864256)]} 0 1
[junit4:junit4] 2> 567373 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437746168134107136)]} 0 1
[junit4:junit4] 2> 567373 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1437746168134107136)]} 0 4
[junit4:junit4] 2> 567374 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[8]} 0 7
[junit4:junit4] 2> 567378 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[9 (1437746168142495744)]} 0 1
[junit4:junit4] 2> 567384 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437746168145641472)]} 0 1
[junit4:junit4] 2> 567384 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[9 (1437746168145641472)]} 0 4
[junit4:junit4] 2> 567388 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[10 (1437746168152981504)]} 0 1
[junit4:junit4] 2> 567396 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437746168158224384)]} 0 1
[junit4:junit4] 2> 567397 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1437746168158224384)]} 0 4
[junit4:junit4] 2> 567397 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[10]} 0 7
[junit4:junit4] 2> 567401 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[11 (1437746168166612992)]} 0 1
[junit4:junit4] 2> 567409 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437746168171855872)]} 0 1
[junit4:junit4] 2> 567409 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1437746168171855872)]} 0 4
[junit4:junit4] 2> 567410 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[11]} 0 7
[junit4:junit4] 2> 567414 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[12 (1437746168180244480)]} 0 1
[junit4:junit4] 2> 567423 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437746168186535936)]} 0 1
[junit4:junit4] 2> 567423 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1437746168186535936)]} 0 4
[junit4:junit4] 2> 567424 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[12]} 0 7
[junit4:junit4] 2> 567428 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[13 (1437746168194924544)]} 0 1
[junit4:junit4] 2> 567434 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437746168198070272)]} 0 1
[junit4:junit4] 2> 567434 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[13 (1437746168198070272)]} 0 4
[junit4:junit4] 2> 567438 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[14 (1437746168205410304)]} 0 1
[junit4:junit4] 2> 567445 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437746168209604608)]} 0 1
[junit4:junit4] 2> 567445 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[14 (1437746168209604608)]} 0 4
[junit4:junit4] 2> 567449 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[15 (1437746168216944640)]} 0 1
[junit4:junit4] 2> 567456 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437746168221138944)]} 0 1
[junit4:junit4] 2> 567456 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[15 (1437746168221138944)]} 0 4
[junit4:junit4] 2> 567460 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[16 (1437746168228478976)]} 0 1
[junit4:junit4] 2> 567467 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437746168232673280)]} 0 1
[junit4:junit4] 2> 567467 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[16 (1437746168232673280)]} 0 4
[junit4:junit4] 2> 567471 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[17 (1437746168240013312)]} 0 1
[junit4:junit4] 2> 567479 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437746168245256192)]} 0 1
[junit4:junit4] 2> 567479 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1437746168245256192)]} 0 4
[junit4:junit4] 2> 567480 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4] 2> 567484 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[18 (1437746168253644800)]} 0 1
[junit4:junit4] 2> 567493 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437746168259936256)]} 0 1
[junit4:junit4] 2> 567493 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1437746168259936256)]} 0 4
[junit4:junit4] 2> 567494 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[18]} 0 7
[junit4:junit4] 2> 567498 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[19 (1437746168268324864)]} 0 1
[junit4:junit4] 2> 567504 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437746168271470592)]} 0 1
[junit4:junit4] 2> 567504 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[19 (1437746168271470592)]} 0 4
[junit4:junit4] 2> 567508 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[20 (1437746168278810624)]} 0 1
[junit4:junit4] 2> 567516 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437746168284053504)]} 0 1
[junit4:junit4] 2> 567516 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1437746168284053504)]} 0 4
[junit4:junit4] 2> 567517 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[20]} 0 7
[junit4:junit4] 2> 567521 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[21 (1437746168292442112)]} 0 1
[junit4:junit4] 2> 567530 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437746168298733568)]} 0 1
[junit4:junit4] 2> 567530 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1437746168298733568)]} 0 4
[junit4:junit4] 2> 567531 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[21]} 0 7
[junit4:junit4] 2> 567535 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[22 (1437746168307122176)]} 0 1
[junit4:junit4] 2> 567542 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437746168311316480)]} 0 1
[junit4:junit4] 2> 567542 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[22 (1437746168311316480)]} 0 4
[junit4:junit4] 2> 567546 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[23 (1437746168318656512)]} 0 1
[junit4:junit4] 2> 567552 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437746168321802240)]} 0 1
[junit4:junit4] 2> 567552 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[23 (1437746168321802240)]} 0 4
[junit4:junit4] 2> 567556 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[24 (1437746168329142272)]} 0 1
[junit4:junit4] 2> 567563 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437746168333336576)]} 0 1
[junit4:junit4] 2> 567563 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[24 (1437746168333336576)]} 0 4
[junit4:junit4] 2> 567567 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[25 (1437746168340676608)]} 0 1
[junit4:junit4] 2> 567575 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437746168345919488)]} 0 1
[junit4:junit4] 2> 567575 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1437746168345919488)]} 0 4
[junit4:junit4] 2> 567576 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4] 2> 567580 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[26 (1437746168354308096)]} 0 1
[junit4:junit4] 2> 567589 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437746168360599552)]} 0 1
[junit4:junit4] 2> 567589 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1437746168360599552)]} 0 4
[junit4:junit4] 2> 567590 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[26]} 0 7
[junit4:junit4] 2> 567594 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[27 (1437746168368988160)]} 0 1
[junit4:junit4] 2> 567603 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437746168375279616)]} 0 1
[junit4:junit4] 2> 567603 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437746168375279616)]} 0 4
[junit4:junit4] 2> 567604 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[27]} 0 7
[junit4:junit4] 2> 567608 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[28 (1437746168383668224)]} 0 1
[junit4:junit4] 2> 567616 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437746168388911104)]} 0 1
[junit4:junit4] 2> 567616 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1437746168388911104)]} 0 4
[junit4:junit4] 2> 567617 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 567621 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[29 (1437746168397299712)]} 0 1
[junit4:junit4] 2> 567627 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437746168400445440)]} 0 1
[junit4:junit4] 2> 567627 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[29 (1437746168400445440)]} 0 4
[junit4:junit4] 2> 567631 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[30 (1437746168407785472)]} 0 1
[junit4:junit4] 2> 567638 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437746168411979776)]} 0 1
[junit4:junit4] 2> 567638 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[30 (1437746168411979776)]} 0 4
[junit4:junit4] 2> 567642 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[31 (1437746168419319808)]} 0 1
[junit4:junit4] 2> 567650 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437746168424562688)]} 0 1
[junit4:junit4] 2> 567650 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437746168424562688)]} 0 4
[junit4:junit4] 2> 567651 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4] 2> 567655 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[32 (1437746168432951296)]} 0 1
[junit4:junit4] 2> 567661 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437746168436097024)]} 0 1
[junit4:junit4] 2> 567661 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[32 (1437746168436097024)]} 0 4
[junit4:junit4] 2> 567665 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[33 (1437746168443437056)]} 0 1
[junit4:junit4] 2> 567674 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437746168449728512)]} 0 1
[junit4:junit4] 2> 567674 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1437746168449728512)]} 0 4
[junit4:junit4] 2> 567675 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[33]} 0 7
[junit4:junit4] 2> 567679 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[34 (1437746168458117120)]} 0 1
[junit4:junit4] 2> 567687 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437746168463360000)]} 0 1
[junit4:junit4] 2> 567687 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1437746168463360000)]} 0 4
[junit4:junit4] 2> 567688 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[34]} 0 7
[junit4:junit4] 2> 567692 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[35 (1437746168471748608)]} 0 1
[junit4:junit4] 2> 567698 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437746168474894336)]} 0 1
[junit4:junit4] 2> 567698 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[35 (1437746168474894336)]} 0 4
[junit4:junit4] 2> 567702 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[36 (1437746168482234368)]} 0 1
[junit4:junit4] 2> 567708 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437746168485380096)]} 0 1
[junit4:junit4] 2> 567708 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[36 (1437746168485380096)]} 0 4
[junit4:junit4] 2> 567712 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[37 (1437746168492720128)]} 0 1
[junit4:junit4] 2> 567721 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437746168499011584)]} 0 1
[junit4:junit4] 2> 567721 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1437746168499011584)]} 0 4
[junit4:junit4] 2> 567722 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4] 2> 567726 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[38 (1437746168507400192)]} 0 1
[junit4:junit4] 2> 567734 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1437746168512643072)]} 0 1
[junit4:junit4] 2> 567734 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1437746168512643072)]} 0 4
[junit4:junit4] 2> 567735 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[38]} 0 7
[junit4:junit4] 2> 567739 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[39 (1437746168521031680)]} 0 1
[junit4:junit4] 2> 567746 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1437746168525225984)]} 0 1
[junit4:junit4] 2> 567746 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[39 (1437746168525225984)]} 0 4
[junit4:junit4] 2> 567750 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[40 (1437746168532566016)]} 0 1
[junit4:junit4] 2> 567756 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1437746168535711744)]} 0 1
[junit4:junit4] 2> 567756 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[40 (1437746168535711744)]} 0 4
[junit4:junit4] 2> 567760 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[41 (1437746168543051776)]} 0 1
[junit4:junit4] 2> 567769 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1437746168549343232)]} 0 1
[junit4:junit4] 2> 567769 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1437746168549343232)]} 0 4
[junit4:junit4] 2> 567770 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[41]} 0 7
[junit4:junit4] 2> 567774 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[42 (1437746168557731840)]} 0 1
[junit4:junit4] 2> 567782 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1437746168562974720)]} 0 1
[junit4:junit4] 2> 567782 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1437746168562974720)]} 0 4
[junit4:junit4] 2> 567783 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[42]} 0 7
[junit4:junit4] 2> 567787 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[43 (1437746168571363328)]} 0 1
[junit4:junit4] 2> 567796 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1437746168577654784)]} 0 1
[junit4:junit4] 2> 567796 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1437746168577654784)]} 0 4
[junit4:junit4] 2> 567797 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[43]} 0 7
[junit4:junit4] 2> 567801 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[44 (1437746168586043392)]} 0 1
[junit4:junit4] 2> 567810 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1437746168592334848)]} 0 1
[junit4:junit4] 2> 567810 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1437746168592334848)]} 0 4
[junit4:junit4] 2> 567811 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[44]} 0 7
[junit4:junit4] 2> 567815 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[45 (1437746168600723456)]} 0 1
[junit4:junit4] 2> 567821 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1437746168603869184)]} 0 1
[junit4:junit4] 2> 567821 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[45 (1437746168603869184)]} 0 4
[junit4:junit4] 2> 567825 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[46 (1437746168611209216)]} 0 1
[junit4:junit4] 2> 567831 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1437746168614354944)]} 0 1
[junit4:junit4] 2> 567831 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[46 (1437746168614354944)]} 0 4
[junit4:junit4] 2> 567835 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[47 (1437746168621694976)]} 0 1
[junit4:junit4] 2> 567842 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1437746168625889280)]} 0 1
[junit4:junit4] 2> 567843 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[47 (1437746168625889280)]} 0 5
[junit4:junit4] 2> 567847 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[48 (1437746168634277888)]} 0 1
[junit4:junit4] 2> 567854 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1437746168638472192)]} 0 1
[junit4:junit4] 2> 567854 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[48 (1437746168638472192)]} 0 4
[junit4:junit4] 2> 567858 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[49 (1437746168645812224)]} 0 1
[junit4:junit4] 2> 567864 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1437746168648957952)]} 0 1
[junit4:junit4] 2> 567864 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[49 (1437746168648957952)]} 0 4
[junit4:junit4] 2> 567868 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[50 (1437746168656297984)]} 0 1
[junit4:junit4] 2> 567874 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1437746168659443712)]} 0 1
[junit4:junit4] 2> 567875 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[50 (1437746168659443712)]} 0 5
[junit4:junit4] 2> 567879 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[51 (1437746168667832320)]} 0 1
[junit4:junit4] 2> 567885 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1437746168670978048)]} 0 1
[junit4:junit4] 2> 567885 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[51 (1437746168670978048)]} 0 4
[junit4:junit4] 2> 567889 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[52 (1437746168678318080)]} 0 1
[junit4:junit4] 2> 567895 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1437746168681463808)]} 0 1
[junit4:junit4] 2> 567895 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[52 (1437746168681463808)]} 0 4
[junit4:junit4] 2> 567899 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[53 (1437746168688803840)]} 0 1
[junit4:junit4] 2> 567905 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1437746168691949568)]} 0 1
[junit4:junit4] 2> 567906 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[53 (1437746168691949568)]} 0 5
[junit4:junit4] 2> 567910 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[54 (1437746168700338176)]} 0 1
[junit4:junit4] 2> 567919 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1437746168706629632)]} 0 1
[junit4:junit4] 2> 567920 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1437746168706629632)]} 0 5
[junit4:junit4] 2> 567921 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[54]} 0 8
[junit4:junit4] 2> 567924 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[55 (1437746168716066816)]} 0 1
[junit4:junit4] 2> 567931 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1437746168719212544)]} 0 1
[junit4:junit4] 2> 567932 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[55 (1437746168719212544)]} 0 5
[junit4:junit4] 2> 567936 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[56 (1437746168727601152)]} 0 1
[junit4:junit4] 2> 567946 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1437746168734941184)]} 0 1
[junit4:junit4] 2> 567947 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1437746168734941184)]} 0 5
[junit4:junit4] 2> 567947 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[56]} 0 8
[junit4:junit4] 2> 567951 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[57 (1437746168743329792)]} 0 1
[junit4:junit4] 2> 567960 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1437746168749621248)]} 0 1
[junit4:junit4] 2> 567961 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1437746168749621248)]} 0 5
[junit4:junit4] 2> 567962 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[57]} 0 8
[junit4:junit4] 2> 567966 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[58 (1437746168759058432)]} 0 1
[junit4:junit4] 2> 567975 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1437746168765349888)]} 0 1
[junit4:junit4] 2> 567976 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1437746168765349888)]} 0 5
[junit4:junit4] 2> 567977 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[58]} 0 8
[junit4:junit4] 2> 567981 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[59 (1437746168774787072)]} 0 1
[junit4:junit4] 2> 567988 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1437746168778981376)]} 0 1
[junit4:junit4] 2> 567989 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[59 (1437746168778981376)]} 0 5
[junit4:junit4] 2> 567993 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[60 (1437746168787369984)]} 0 1
[junit4:junit4] 2> 568002 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1437746168793661440)]} 0 1
[junit4:junit4] 2> 568003 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1437746168793661440)]} 0 5
[junit4:junit4] 2> 568004 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[60]} 0 8
[junit4:junit4] 2> 568008 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[61 (1437746168803098624)]} 0 1
[junit4:junit4] 2> 568018 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1437746168810438656)]} 0 1
[junit4:junit4] 2> 568018 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1437746168810438656)]} 0 4
[junit4:junit4] 2> 568019 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[61]} 0 8
[junit4:junit4] 2> 568023 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[62 (1437746168818827264)]} 0 1
[junit4:junit4] 2> 568032 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1437746168825118720)]} 0 1
[junit4:junit4] 2> 568032 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[62 (1437746168825118720)]} 0 4
[junit4:junit4] 2> 568033 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[62]} 0 7
[junit4:junit4] 2> 568037 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[63 (1437746168833507328)]} 0 1
[junit4:junit4] 2> 568044 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1437746168837701632)]} 0 1
[junit4:junit4] 2> 568045 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[63 (1437746168837701632)]} 0 5
[junit4:junit4] 2> 568049 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[64 (1437746168846090240)]} 0 1
[junit4:junit4] 2> 568056 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1437746168850284544)]} 0 1
[junit4:junit4] 2> 568057 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[64 (1437746168850284544)]} 0 5
[junit4:junit4] 2> 568061 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[65 (1437746168858673152)]} 0 1
[junit4:junit4] 2> 568071 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1437746168866013184)]} 0 1
[junit4:junit4] 2> 568072 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[65 (1437746168866013184)]} 0 5
[junit4:junit4] 2> 568072 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[65]} 0 8
[junit4:junit4] 2> 568076 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[66 (1437746168874401792)]} 0 1
[junit4:junit4] 2> 568086 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[66 (1437746168881741824)]} 0 1
[junit4:junit4] 2> 568087 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[66 (1437746168881741824)]} 0 5
[junit4:junit4] 2> 568088 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[66]} 0 9
[junit4:junit4] 2> 568091 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[67 (1437746168891179008)]} 0 1
[junit4:junit4] 2> 568098 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[67 (1437746168894324736)]} 0 1
[junit4:junit4] 2> 568099 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[67 (1437746168894324736)]} 0 5
[junit4:junit4] 2> 568103 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[68 (1437746168902713344)]} 0 1
[junit4:junit4] 2> 568110 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[68 (1437746168906907648)]} 0 1
[junit4:junit4] 2> 568111 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[68 (1437746168906907648)]} 0 5
[junit4:junit4] 2> 568115 T872 C162 P27612 oasup.L
[...truncated too long message...]
"shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 773186 T1013 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:62951_y_q_collection1_shard1_0_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1_0",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":null,
[junit4:junit4] 2> "core":"collection1_shard1_0_replica1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 773197 T1013 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:62951_y_q_collection1_shard1_1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1_1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":null,
[junit4:junit4] 2> "core":"collection1_shard1_1_replica1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4] 2> 773209 T860 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 32634
[junit4:junit4] 2> 773209 T860 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1242169106
[junit4:junit4] 2> 773222 T951 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 773223 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 774222 T860 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 774222 T860 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 774225 T860 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5b967893
[junit4:junit4] 2> 774227 T860 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=861,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 774228 T860 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 774228 T860 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 774229 T860 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 774231 T860 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 774232 T860 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 774232 T860 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4;done=false>>]
[junit4:junit4] 2> 774233 T860 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4
[junit4:junit4] 2> 774233 T860 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index;done=false>>]
[junit4:junit4] 2> 774234 T860 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index
[junit4:junit4] 2> 774235 T1013 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89859133664067596-127.0.0.1:32634_y_q-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 774236 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 774236 T951 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 774237 T860 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/y_q,null}
[junit4:junit4] 2> 774237 T951 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 774309 T860 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 774311 T860 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55935 55935
[junit4:junit4] 2> 774454 T934 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 774583 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 774583 T934 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 774584 T860 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 27612
[junit4:junit4] 2> !!!! WARNING: best effort to remove /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910 FAILED !!!!!
[junit4:junit4] 2> 774585 T860 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 774586 T860 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55935 55935
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=94C1FB14CE220E9A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=be -Dtests.timezone=America/Kralendijk -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 236s J1 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([94C1FB14CE220E9A:1527750CB97D6EA6]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 774744 T860 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 235538 T859 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=be, timezone=America/Kralendijk
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=1,free=160813512,total=387973120
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ShowFileRequestHandlerTest, StandardRequestHandlerTest, SuggesterWFSTTest, FieldMutatingUpdateProcessorTest, TestMaxScoreQueryParser, ShardRoutingCustomTest, TestSolrXMLSerializer, CollectionsAPIDistributedZkTest, ShardSplitTest, ShardRoutingTest, SolrCoreCheckLockOnStartupTest, TestBadConfig, DOMUtilTest, DocValuesMultiTest, PrimUtilsTest, TestLRUCache, SuggesterTSTTest, TestDynamicFieldResource, IndexSchemaRuntimeFieldTest, TestHashPartitioner, TestFieldTypeCollectionResource, CacheHeaderTest, CSVRequestHandlerTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 236.33s, 1 test, 1 error <<< FAILURES!
[...truncated 731 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 1 error, 13 ignored (7 assumptions)
Total time: 51 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1327 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1327/
1 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:41573/_yfv/z returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:41573/_yfv/z returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([4689860FC4A9F9BB:C76F0817B3F69987]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9668 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 1162733 T1639 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_yfv/z
[junit4:junit4] 2> 1162738 T1639 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1371091874556
[junit4:junit4] 2> 1162740 T1639 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1162741 T1640 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1162842 T1639 oasc.ZkTestServer.run start zk server on port:12489
[junit4:junit4] 2> 1162843 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1162867 T1646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38834033 name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1162867 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1162868 T1639 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1162874 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1162904 T1648 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54ca9109 name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1162905 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1162905 T1639 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1162911 T1639 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1162914 T1639 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1162917 T1639 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1162921 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1162922 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1162967 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1162968 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1163073 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1163074 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1163084 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1163085 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1163115 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1163116 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1163119 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1163120 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1163124 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1163125 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1163164 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1163165 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1163168 T1639 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1163169 T1639 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1163441 T1639 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1163446 T1639 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21875
[junit4:junit4] 2> 1163447 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1163447 T1639 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1163448 T1639 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991
[junit4:junit4] 2> 1163448 T1639 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/solr.xml
[junit4:junit4] 2> 1163449 T1639 oasc.CoreContainer.<init> New CoreContainer 1663887369
[junit4:junit4] 2> 1163449 T1639 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/'
[junit4:junit4] 2> 1163450 T1639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/'
[junit4:junit4] 2> 1163577 T1639 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1163578 T1639 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1163579 T1639 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1163579 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1163580 T1639 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1163581 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1163581 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1163582 T1639 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1163582 T1639 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1163583 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1163602 T1639 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1163603 T1639 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12489/solr
[junit4:junit4] 2> 1163603 T1639 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1163604 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1163607 T1659 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dfc2f08 name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1163608 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1163611 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1163624 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1163626 T1661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5055d6ac name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1163627 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1163630 T1639 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1163639 T1639 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1163643 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1163646 T1639 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21875__yfv%2Fz
[junit4:junit4] 2> 1163653 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21875__yfv%2Fz
[junit4:junit4] 2> 1163657 T1639 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1163670 T1639 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1163680 T1639 oasc.Overseer.start Overseer (id=89855877091426307-127.0.0.1:21875__yfv%2Fz-n_0000000000) starting
[junit4:junit4] 2> 1163685 T1639 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1163699 T1663 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1163700 T1639 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1163703 T1639 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1163705 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1163708 T1662 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1163712 T1664 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1163713 T1664 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1165214 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1165215 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21875__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21875/_yfv/z"}
[junit4:junit4] 2> 1165215 T1662 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1165215 T1662 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1165236 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1165716 T1664 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/collection1
[junit4:junit4] 2> 1165716 T1664 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1165717 T1664 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1165717 T1664 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1165719 T1664 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/collection1/'
[junit4:junit4] 2> 1165721 T1664 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/collection1/lib/README' to classloader
[junit4:junit4] 2> 1165721 T1664 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1165774 T1664 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1165834 T1664 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1165936 T1664 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1165942 T1664 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1166523 T1664 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1166527 T1664 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1166529 T1664 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1166535 T1664 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1166564 T1664 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1166565 T1664 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371091874991/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data/
[junit4:junit4] 2> 1166565 T1664 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@303be7e8
[junit4:junit4] 2> 1166566 T1664 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1166566 T1664 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data
[junit4:junit4] 2> 1166567 T1664 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data/index/
[junit4:junit4] 2> 1166567 T1664 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1166567 T1664 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data/index
[junit4:junit4] 2> 1166572 T1664 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32988356 lockFactory=org.apache.lucene.store.NativeFSLockFactory@243a18c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1166573 T1664 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1166575 T1664 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1166576 T1664 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1166577 T1664 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1166577 T1664 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1166578 T1664 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1166578 T1664 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1166578 T1664 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1166579 T1664 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1166579 T1664 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1166596 T1664 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1166603 T1664 oass.SolrIndexSearcher.<init> Opening Searcher@2494649b main
[junit4:junit4] 2> 1166603 T1664 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/control/data/tlog
[junit4:junit4] 2> 1166605 T1664 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1166605 T1664 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1166608 T1665 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2494649b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1166610 T1664 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1166610 T1664 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21875/_yfv/z collection:control_collection shard:shard1
[junit4:junit4] 2> 1166611 T1664 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1166636 T1664 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1166639 T1664 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1166640 T1664 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1166640 T1664 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21875/_yfv/z/collection1/
[junit4:junit4] 2> 1166640 T1664 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1166640 T1664 oasc.SyncStrategy.syncToMe http://127.0.0.1:21875/_yfv/z/collection1/ has no replicas
[junit4:junit4] 2> 1166641 T1664 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21875/_yfv/z/collection1/
[junit4:junit4] 2> 1166641 T1664 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1166741 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1166757 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1166801 T1664 oasc.ZkController.register We are http://127.0.0.1:21875/_yfv/z/collection1/ and leader is http://127.0.0.1:21875/_yfv/z/collection1/
[junit4:junit4] 2> 1166801 T1664 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21875/_yfv/z
[junit4:junit4] 2> 1166801 T1664 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1166802 T1664 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1166802 T1664 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1166804 T1664 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1166805 T1639 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1166806 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1166806 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1166825 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1166827 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1166829 T1668 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48ee6732 name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1166830 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1166832 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1166835 T1639 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1167104 T1639 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1167108 T1639 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41573
[junit4:junit4] 2> 1167109 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1167109 T1639 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1167110 T1639 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652
[junit4:junit4] 2> 1167110 T1639 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/solr.xml
[junit4:junit4] 2> 1167111 T1639 oasc.CoreContainer.<init> New CoreContainer 976935945
[junit4:junit4] 2> 1167111 T1639 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/'
[junit4:junit4] 2> 1167112 T1639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/'
[junit4:junit4] 2> 1167239 T1639 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1167240 T1639 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1167241 T1639 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1167241 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1167242 T1639 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1167242 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1167243 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1167243 T1639 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1167244 T1639 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1167244 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1167263 T1639 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1167264 T1639 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12489/solr
[junit4:junit4] 2> 1167265 T1639 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1167266 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1167269 T1679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ea51205 name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1167269 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1167282 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1167296 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1167298 T1681 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@622e2687 name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1167299 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1167310 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1168264 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1168265 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:21875__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21875__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21875/_yfv/z"}
[junit4:junit4] 2> 1168284 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1168284 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1168284 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1168314 T1639 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41573__yfv%2Fz
[junit4:junit4] 2> 1168316 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41573__yfv%2Fz
[junit4:junit4] 2> 1168319 T1668 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1168319 T1661 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1168320 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1168320 T1681 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1168320 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1168320 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1168332 T1682 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1168332 T1682 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1169795 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1169796 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41573__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41573/_yfv/z"}
[junit4:junit4] 2> 1169796 T1662 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1169796 T1662 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1169813 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1169813 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1169813 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1170336 T1682 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/collection1
[junit4:junit4] 2> 1170336 T1682 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1170337 T1682 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1170337 T1682 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1170339 T1682 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/collection1/'
[junit4:junit4] 2> 1170341 T1682 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1170341 T1682 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/collection1/lib/README' to classloader
[junit4:junit4] 2> 1170394 T1682 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1170454 T1682 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1170556 T1682 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1170563 T1682 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1171139 T1682 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1171143 T1682 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1171146 T1682 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1171151 T1682 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1171181 T1682 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1171181 T1682 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371091878652/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1/
[junit4:junit4] 2> 1171182 T1682 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@303be7e8
[junit4:junit4] 2> 1171182 T1682 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1171183 T1682 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1
[junit4:junit4] 2> 1171183 T1682 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1/index/
[junit4:junit4] 2> 1171184 T1682 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1171185 T1682 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1/index
[junit4:junit4] 2> 1171188 T1682 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1171189 T1682 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1171192 T1682 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1171192 T1682 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1171193 T1682 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1171194 T1682 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1171194 T1682 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1171195 T1682 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1171195 T1682 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1171196 T1682 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1171196 T1682 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1171213 T1682 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1171220 T1682 oass.SolrIndexSearcher.<init> Opening Searcher@39dc07d1 main
[junit4:junit4] 2> 1171221 T1682 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty1/tlog
[junit4:junit4] 2> 1171222 T1682 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1171222 T1682 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1171226 T1683 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39dc07d1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1171228 T1682 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1171229 T1682 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41573/_yfv/z collection:collection1 shard:shard1
[junit4:junit4] 2> 1171230 T1682 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1171251 T1682 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1171254 T1682 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1171255 T1682 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1171255 T1682 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41573/_yfv/z/collection1/
[junit4:junit4] 2> 1171255 T1682 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1171256 T1682 oasc.SyncStrategy.syncToMe http://127.0.0.1:41573/_yfv/z/collection1/ has no replicas
[junit4:junit4] 2> 1171256 T1682 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41573/_yfv/z/collection1/
[junit4:junit4] 2> 1171256 T1682 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1171319 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1171335 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1171335 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1171335 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1171365 T1682 oasc.ZkController.register We are http://127.0.0.1:41573/_yfv/z/collection1/ and leader is http://127.0.0.1:41573/_yfv/z/collection1/
[junit4:junit4] 2> 1171365 T1682 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41573/_yfv/z
[junit4:junit4] 2> 1171365 T1682 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1171365 T1682 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1171366 T1682 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1171368 T1682 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1171369 T1639 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1171370 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1171370 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1171652 T1639 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1171656 T1639 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41374
[junit4:junit4] 2> 1171656 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1171657 T1639 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1171658 T1639 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207
[junit4:junit4] 2> 1171658 T1639 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/solr.xml
[junit4:junit4] 2> 1171659 T1639 oasc.CoreContainer.<init> New CoreContainer 597554431
[junit4:junit4] 2> 1171659 T1639 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/'
[junit4:junit4] 2> 1171660 T1639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/'
[junit4:junit4] 2> 1171787 T1639 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1171788 T1639 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1171789 T1639 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1171790 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1171790 T1639 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1171791 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1171791 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1171792 T1639 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1171792 T1639 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1171793 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1171812 T1639 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1171813 T1639 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12489/solr
[junit4:junit4] 2> 1171813 T1639 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1171814 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1171817 T1695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1827ec0d name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1171818 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1171831 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1171845 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1171847 T1697 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19a6b5a name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1171848 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1171854 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1172843 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1172844 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41573__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41573__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41573/_yfv/z"}
[junit4:junit4] 2> 1172867 T1639 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41374__yfv%2Fz
[junit4:junit4] 2> 1172870 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41374__yfv%2Fz
[junit4:junit4] 2> 1172885 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1172885 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1172885 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1172885 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1172902 T1681 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1172902 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1172903 T1668 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1172903 T1697 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1172903 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1172968 T1661 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1172968 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1172973 T1698 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1172974 T1698 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1174405 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1174406 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41374__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41374/_yfv/z"}
[junit4:junit4] 2> 1174406 T1662 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1174407 T1662 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1174429 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1174429 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1174429 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1174429 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1174978 T1698 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/collection1
[junit4:junit4] 2> 1174978 T1698 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1174979 T1698 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1174979 T1698 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1174981 T1698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/collection1/'
[junit4:junit4] 2> 1174983 T1698 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1174983 T1698 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/collection1/lib/README' to classloader
[junit4:junit4] 2> 1175036 T1698 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1175097 T1698 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1175198 T1698 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1175205 T1698 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1175781 T1698 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1175785 T1698 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1175788 T1698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1175793 T1698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1175822 T1698 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1175823 T1698 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371091883207/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2/
[junit4:junit4] 2> 1175823 T1698 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@303be7e8
[junit4:junit4] 2> 1175824 T1698 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1175825 T1698 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2
[junit4:junit4] 2> 1175825 T1698 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2/index/
[junit4:junit4] 2> 1175825 T1698 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1175826 T1698 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2/index
[junit4:junit4] 2> 1175830 T1698 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1175830 T1698 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1175833 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1175833 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1175834 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1175835 T1698 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1175835 T1698 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1175835 T1698 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1175836 T1698 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1175836 T1698 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1175837 T1698 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1175852 T1698 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1175859 T1698 oass.SolrIndexSearcher.<init> Opening Searcher@59280df0 main
[junit4:junit4] 2> 1175859 T1698 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty2/tlog
[junit4:junit4] 2> 1175860 T1698 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1175861 T1698 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1175865 T1699 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59280df0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1175867 T1698 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1175867 T1698 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41374/_yfv/z collection:collection1 shard:shard2
[junit4:junit4] 2> 1175868 T1698 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1175893 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1175896 T1698 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1175896 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1175896 T1698 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41374/_yfv/z/collection1/
[junit4:junit4] 2> 1175897 T1698 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1175897 T1698 oasc.SyncStrategy.syncToMe http://127.0.0.1:41374/_yfv/z/collection1/ has no replicas
[junit4:junit4] 2> 1175897 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41374/_yfv/z/collection1/
[junit4:junit4] 2> 1175898 T1698 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1175935 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1175943 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1175943 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1175943 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1175943 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1175955 T1698 oasc.ZkController.register We are http://127.0.0.1:41374/_yfv/z/collection1/ and leader is http://127.0.0.1:41374/_yfv/z/collection1/
[junit4:junit4] 2> 1175955 T1698 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41374/_yfv/z
[junit4:junit4] 2> 1175955 T1698 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1175955 T1698 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1175956 T1698 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1175958 T1698 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1175959 T1639 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1175960 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1175960 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1176228 T1639 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1176231 T1639 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41239
[junit4:junit4] 2> 1176231 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1176232 T1639 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1176232 T1639 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795
[junit4:junit4] 2> 1176233 T1639 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/solr.xml
[junit4:junit4] 2> 1176233 T1639 oasc.CoreContainer.<init> New CoreContainer 1358864396
[junit4:junit4] 2> 1176234 T1639 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/'
[junit4:junit4] 2> 1176235 T1639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/'
[junit4:junit4] 2> 1176353 T1639 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1176354 T1639 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1176355 T1639 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1176355 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1176356 T1639 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1176356 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1176357 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1176357 T1639 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1176358 T1639 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1176358 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1176376 T1639 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1176376 T1639 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12489/solr
[junit4:junit4] 2> 1176377 T1639 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1176378 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1176381 T1711 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@365be4bd name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1176382 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1176397 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1176411 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1176414 T1713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27506b8e name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1176414 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1176426 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1177430 T1639 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41239__yfv%2Fz
[junit4:junit4] 2> 1177433 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41239__yfv%2Fz
[junit4:junit4] 2> 1177437 T1668 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1177437 T1661 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1177437 T1697 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1177437 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177437 T1713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1177437 T1681 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1177438 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177438 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177439 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177452 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1177453 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41374__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41374__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41374/_yfv/z"}
[junit4:junit4] 2> 1177456 T1714 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1177456 T1714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1177463 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41239__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41239/_yfv/z"}
[junit4:junit4] 2> 1177463 T1662 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1177463 T1662 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1177473 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177474 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177473 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177473 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1177473 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1178463 T1714 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/collection1
[junit4:junit4] 2> 1178463 T1714 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1178464 T1714 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1178465 T1714 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1178466 T1714 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/collection1/'
[junit4:junit4] 2> 1178468 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/collection1/lib/README' to classloader
[junit4:junit4] 2> 1178469 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1178522 T1714 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1178582 T1714 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1178684 T1714 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1178690 T1714 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1179268 T1714 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1179272 T1714 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1179275 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1179280 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1179310 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1179311 T1714 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371091887795/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/
[junit4:junit4] 2> 1179311 T1714 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@303be7e8
[junit4:junit4] 2> 1179312 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1179312 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3
[junit4:junit4] 2> 1179313 T1714 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index/
[junit4:junit4] 2> 1179313 T1714 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1179314 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index
[junit4:junit4] 2> 1179318 T1714 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@490d0f75 lockFactory=org.apache.lucene.store.NativeFSLockFactory@564a5516),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1179318 T1714 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1179321 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1179322 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1179322 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1179323 T1714 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1179324 T1714 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1179324 T1714 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1179324 T1714 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1179325 T1714 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1179326 T1714 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1179342 T1714 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1179349 T1714 oass.SolrIndexSearcher.<init> Opening Searcher@717974b1 main
[junit4:junit4] 2> 1179350 T1714 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/tlog
[junit4:junit4] 2> 1179351 T1714 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1179351 T1714 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1179355 T1715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@717974b1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1179357 T1714 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1179358 T1714 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41239/_yfv/z collection:collection1 shard:shard1
[junit4:junit4] 2> 1179362 T1714 oasc.ZkController.register We are http://127.0.0.1:41239/_yfv/z/collection1/ and leader is http://127.0.0.1:41573/_yfv/z/collection1/
[junit4:junit4] 2> 1179362 T1714 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41239/_yfv/z
[junit4:junit4] 2> 1179363 T1714 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1179363 T1714 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C825 name=collection1 org.apache.solr.core.SolrCore@6fb4a132 url=http://127.0.0.1:41239/_yfv/z/collection1 node=127.0.0.1:41239__yfv%2Fz C825_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:41239__yfv%2Fz, base_url=http://127.0.0.1:41239/_yfv/z}
[junit4:junit4] 2> 1179363 T1716 C825 P41239 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1179364 T1716 C825 P41239 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1179364 T1714 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1179364 T1716 C825 P41239 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1179365 T1716 C825 P41239 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1179365 T1639 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1179366 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1179367 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1179367 T1716 C825 P41239 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1179396 T1671 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:41239__yfv%2Fz_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1179662 T1639 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1179665 T1639 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:64758
[junit4:junit4] 2> 1179666 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1179666 T1639 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1179667 T1639 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212
[junit4:junit4] 2> 1179667 T1639 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/solr.xml
[junit4:junit4] 2> 1179668 T1639 oasc.CoreContainer.<init> New CoreContainer 1049166073
[junit4:junit4] 2> 1179668 T1639 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/'
[junit4:junit4] 2> 1179669 T1639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/'
[junit4:junit4] 2> 1179799 T1639 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1179800 T1639 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1179800 T1639 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1179801 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1179801 T1639 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1179802 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1179802 T1639 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1179803 T1639 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1179803 T1639 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1179804 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1179823 T1639 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1179824 T1639 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12489/solr
[junit4:junit4] 2> 1179824 T1639 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1179826 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1179829 T1728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cbb9460 name:ZooKeeperConnection Watcher:127.0.0.1:12489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1179830 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1179851 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1179865 T1639 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1179867 T1730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d0c777c name:ZooKeeperConnection Watcher:127.0.0.1:12489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1179867 T1639 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1179879 T1639 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1180490 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1180491 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41239__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41239__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41239/_yfv/z"}
[junit4:junit4] 2> 1180511 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180511 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180511 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180511 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180511 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180511 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1180883 T1639 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64758__yfv%2Fz
[junit4:junit4] 2> 1180885 T1639 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64758__yfv%2Fz
[junit4:junit4] 2> 1180889 T1730 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180889 T1668 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180889 T1697 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180889 T1661 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180889 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180889 T1713 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180889 T1681 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1180890 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180890 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180890 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180889 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180891 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1180904 T1731 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1180904 T1731 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1181399 T1671 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:41239__yfv%2Fz_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1181399 T1671 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:41239__yfv%252Fz_collection1&state=recovering&nodeName=127.0.0.1:41239__yfv%252Fz&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1182018 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1182019 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:64758__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:64758/_yfv/z"}
[junit4:junit4] 2> 1182019 T1662 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1182019 T1662 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1182039 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182039 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182039 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182039 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182039 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182039 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1182908 T1731 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/collection1
[junit4:junit4] 2> 1182908 T1731 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1182909 T1731 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1182910 T1731 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1182911 T1731 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/collection1/'
[junit4:junit4] 2> 1182913 T1731 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/collection1/lib/README' to classloader
[junit4:junit4] 2> 1182914 T1731 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1182967 T1731 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1183029 T1731 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1183031 T1731 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1183037 T1731 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C826 name=collection1 org.apache.solr.core.SolrCore@6fb4a132 url=http://127.0.0.1:41239/_yfv/z/collection1 node=127.0.0.1:41239__yfv%2Fz C826_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:41239__yfv%2Fz, base_url=http://127.0.0.1:41239/_yfv/z}
[junit4:junit4] 2> 1183401 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41573/_yfv/z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1183401 T1716 C826 P41239 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:41239/_yfv/z START replicas=[http://127.0.0.1:41573/_yfv/z/collection1/] nUpdates=100
[junit4:junit4] 2> 1183402 T1716 C826 P41239 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1183402 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1183403 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1183403 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1183403 T1716 C826 P41239 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1183403 T1716 C826 P41239 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41573/_yfv/z/collection1/. core=collection1
[junit4:junit4] 2> 1183404 T1716 C826 P41239 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C827 name=collection1 org.apache.solr.core.SolrCore@4338f5b6 url=http://127.0.0.1:41573/_yfv/z/collection1 node=127.0.0.1:41573__yfv%2Fz C827_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41573__yfv%2Fz, base_url=http://127.0.0.1:41573/_yfv/z, leader=true}
[junit4:junit4] 2> 1183412 T1672 C827 P41573 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1183419 T1673 C827 P41573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1183423 T1673 C827 P41573 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1183423 T1673 C827 P41573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1183424 T1673 C827 P41573 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1183424 T1673 C827 P41573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1183425 T1673 C827 P41573 oass.SolrIndexSearcher.<init> Opening Searcher@3925807f realtime
[junit4:junit4] 2> 1183425 T1673 C827 P41573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1183426 T1673 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1183427 T1716 C826 P41239 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1183427 T1716 C826 P41239 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1183430 T1674 C827 P41573 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1183430 T1674 C827 P41573 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1183431 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1183431 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1183432 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1183434 T1674 C827 P41573 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1183435 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1183436 T1716 C826 P41239 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index.20130613035135252
[junit4:junit4] 2> 1183436 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4161a6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@560fb9a1) fullCopy=false
[junit4:junit4] 2> 1183439 T1674 C827 P41573 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1183440 T1716 C826 P41239 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1183441 T1716 C826 P41239 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1183442 T1716 C826 P41239 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1183444 T1716 C826 P41239 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@490d0f75 lockFactory=org.apache.lucene.store.NativeFSLockFactory@564a5516),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@490d0f75 lockFactory=org.apache.lucene.store.NativeFSLockFactory@564a5516),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1183445 T1716 C826 P41239 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1183445 T1716 C826 P41239 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1183445 T1716 C826 P41239 oass.SolrIndexSearcher.<init> Opening Searcher@6388f9b2 main
[junit4:junit4] 2> 1183446 T1715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6388f9b2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1183447 T1716 C826 P41239 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index.20130613035135252 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index.20130613035135252;done=true>>]
[junit4:junit4] 2> 1183447 T1716 C826 P41239 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index.20130613035135252
[junit4:junit4] 2> 1183447 T1716 C826 P41239 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty3/index.20130613035135252
[junit4:junit4] 2> 1183448 T1716 C826 P41239 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1183448 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1183448 T1716 C826 P41239 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1183448 T1716 C826 P41239 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1183450 T1716 C826 P41239 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1183545 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1183546 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41239__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41239__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41239/_yfv/z"}
[junit4:junit4] 2> 1183564 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183564 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183564 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183564 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183564 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183564 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1183628 T1731 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1183632 T1731 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1183635 T1731 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1183640 T1731 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1183671 T1731 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1183672 T1731 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371091891212/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/
[junit4:junit4] 2> 1183672 T1731 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@303be7e8
[junit4:junit4] 2> 1183673 T1731 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1183674 T1731 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4
[junit4:junit4] 2> 1183674 T1731 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index/
[junit4:junit4] 2> 1183674 T1731 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1183675 T1731 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index
[junit4:junit4] 2> 1183679 T1731 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@46442a24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30ebdc76),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1183679 T1731 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1183682 T1731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1183683 T1731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1183683 T1731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1183684 T1731 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1183685 T1731 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1183685 T1731 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1183686 T1731 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1183687 T1731 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1183687 T1731 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1183704 T1731 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1183711 T1731 oass.SolrIndexSearcher.<init> Opening Searcher@3a7ca039 main
[junit4:junit4] 2> 1183711 T1731 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/tlog
[junit4:junit4] 2> 1183713 T1731 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1183713 T1731 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1183717 T1734 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a7ca039 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1183719 T1731 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1183720 T1731 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:64758/_yfv/z collection:collection1 shard:shard2
[junit4:junit4] 2> 1183723 T1731 oasc.ZkController.register We are http://127.0.0.1:64758/_yfv/z/collection1/ and leader is http://127.0.0.1:41374/_yfv/z/collection1/
[junit4:junit4] 2> 1183724 T1731 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64758/_yfv/z
[junit4:junit4] 2> 1183724 T1731 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1183724 T1731 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C828 name=collection1 org.apache.solr.core.SolrCore@24691a9f url=http://127.0.0.1:64758/_yfv/z/collection1 node=127.0.0.1:64758__yfv%2Fz C828_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:64758__yfv%2Fz, base_url=http://127.0.0.1:64758/_yfv/z}
[junit4:junit4] 2> 1183725 T1735 C828 P64758 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1183725 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1183725 T1731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1183726 T1735 C828 P64758 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1183726 T1735 C828 P64758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1183727 T1639 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1183728 T1639 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1183728 T1735 C828 P64758 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1183728 T1639 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1183756 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1183756 T1687 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:64758__yfv%2Fz_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1183757 T1639 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1183758 T1639 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1183759 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1184761 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1185070 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1185071 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:64758__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:64758__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:64758/_yfv/z"}
[junit4:junit4] 2> 1185088 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185089 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185088 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185088 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185088 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185088 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1185759 T1687 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:64758__yfv%2Fz_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1185759 T1687 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:64758__yfv%252Fz_collection1&state=recovering&nodeName=127.0.0.1:64758__yfv%252Fz&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1185764 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1186767 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C828_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:64758__yfv%2Fz, base_url=http://127.0.0.1:64758/_yfv/z}
[junit4:junit4] 2> 1187761 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41374/_yfv/z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1187761 T1735 C828 P64758 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:64758/_yfv/z START replicas=[http://127.0.0.1:41374/_yfv/z/collection1/] nUpdates=100
[junit4:junit4] 2> 1187762 T1735 C828 P64758 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1187762 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1187762 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1187762 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1187763 T1735 C828 P64758 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1187763 T1735 C828 P64758 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41374/_yfv/z/collection1/. core=collection1
[junit4:junit4] 2> 1187764 T1735 C828 P64758 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1187770 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C829 name=collection1 org.apache.solr.core.SolrCore@70be8672 url=http://127.0.0.1:41374/_yfv/z/collection1 node=127.0.0.1:41374__yfv%2Fz C829_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41374__yfv%2Fz, base_url=http://127.0.0.1:41374/_yfv/z, leader=true}
[junit4:junit4] 2> 1187771 T1688 C829 P41374 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1187777 T1689 C829 P41374 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1187779 T1689 C829 P41374 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1187780 T1689 C829 P41374 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1187781 T1689 C829 P41374 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1187781 T1689 C829 P41374 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1187782 T1689 C829 P41374 oass.SolrIndexSearcher.<init> Opening Searcher@48177f47 realtime
[junit4:junit4] 2> 1187782 T1689 C829 P41374 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1187783 T1689 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1187784 T1735 C828 P64758 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1187784 T1735 C828 P64758 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1187786 T1690 C829 P41374 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1187787 T1690 C829 P41374 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1187787 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1187788 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1187788 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1187790 T1690 C829 P41374 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1187791 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1187792 T1735 C828 P64758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index.20130613035139608
[junit4:junit4] 2> 1187792 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@9e21953 lockFactory=org.apache.lucene.store.NativeFSLockFactory@b77fcf2) fullCopy=false
[junit4:junit4] 2> 1187795 T1690 C829 P41374 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1187797 T1735 C828 P64758 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1187797 T1735 C828 P64758 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1187798 T1735 C828 P64758 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1187799 T1735 C828 P64758 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@46442a24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30ebdc76),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@46442a24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30ebdc76),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1187800 T1735 C828 P64758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1187800 T1735 C828 P64758 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1187801 T1735 C828 P64758 oass.SolrIndexSearcher.<init> Opening Searcher@7e393baa main
[junit4:junit4] 2> 1187802 T1734 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e393baa main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1187802 T1735 C828 P64758 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index.20130613035139608 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index.20130613035139608;done=true>>]
[junit4:junit4] 2> 1187803 T1735 C828 P64758 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index.20130613035139608
[junit4:junit4] 2> 1187803 T1735 C828 P64758 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index.20130613035139608
[junit4:junit4] 2> 1187803 T1735 C828 P64758 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1187803 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1187804 T1735 C828 P64758 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1187804 T1735 C828 P64758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1187806 T1735 C828 P64758 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1188096 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1188097 T1662 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:64758__yfv%2Fz_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:64758__yfv%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:64758/_yfv/z"}
[junit4:junit4] 2> 1188124 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188124 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188124 T1661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188124 T1668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188124 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188124 T1681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1188773 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1188775 T1639 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C830 name=collection1 org.apache.solr.core.SolrCore@5b9b4ba url=http://127.0.0.1:21875/_yfv/z/collection1 node=127.0.0.1:21875__yfv%2Fz C830_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:21875__yfv%2Fz, base_url=http://127.0.0.1:21875/_yfv/z, leader=true}
[junit4:junit4] 2> 1188784 T1651 C830 P21875 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1188787 T1651 C830 P21875 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32988356 lockFactory=org.apache.lucene.store.NativeFSLockFactory@243a18c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1188788 T1651 C830 P21875 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1188789 T1651 C830 P21875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32988356 lockFactory=org.apache.lucene.store.NativeFSLockFactory@243a18c5),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32988356 lockFactory=org.apache.lucene.store.NativeFSLockFactory@243a18c5),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1188789 T1651 C830 P21875 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1188790 T1651 C830 P21875 oass.SolrIndexSearcher.<init> Opening Searcher@3ca63d10 main
[junit4:junit4] 2> 1188791 T1651 C830 P21875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1188791 T1665 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ca63d10 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1188792 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 1188805 T1691 C829 P41374 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1188806 T1691 C829 P41374 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c47271a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfc5a4f),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1188807 T1691 C829 P41374 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1188807 T1691 C829 P41374 oass.SolrIndexSearcher.<init> Opening Searcher@641ad633 main
[junit4:junit4] 2> 1188808 T1691 C829 P41374 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1188809 T1699 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@641ad633 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1188809 T1691 C829 P41374 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41573/_yfv/z/collection1/, StdNode: http://127.0.0.1:41239/_yfv/z/collection1/, StdNode: http://127.0.0.1:64758/_yfv/z/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1188813 T1675 C827 P41573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1188814 T1675 C827 P41573 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6411dc52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d65d76a),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1188815 T1675 C827 P41573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1188815 T1675 C827 P41573 oass.SolrIndexSearcher.<init> Opening Searcher@3325a9 main
[junit4:junit4] 2> 1188816 T1675 C827 P41573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1188817 T1683 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3325a9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1188817 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> C826_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41239__yfv%2Fz, base_url=http://127.0.0.1:41239/_yfv/z}
[junit4:junit4] 2> 1188821 T1703 C826 P41239 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C828_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:64758__yfv%2Fz, base_url=http://127.0.0.1:64758/_yfv/z}
[junit4:junit4] 2> 1188821 T1720 C828 P64758 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1188822 T1703 C826 P41239 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@490d0f75 lockFactory=org.apache.lucene.store.NativeFSLockFactory@564a5516),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@490d0f75 lockFactory=org.apache.lucene.store.NativeFSLockFactory@564a5516),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1188823 T1720 C828 P64758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@46442a24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30ebdc76),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@46442a24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30ebdc76),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1188823 T1703 C826 P41239 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1188824 T1720 C828 P64758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1188824 T1703 C826 P41239 oass.SolrIndexSearcher.<init> Opening Searcher@1251ce05 main
[junit4:junit4] 2> 1188825 T1720 C828 P64758 oass.SolrIndexSearcher.<init> Opening Searcher@1487c491 main
[junit4:junit4] 2> 1188825 T1703 C826 P41239 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1188826 T1715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1251ce05 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1188826 T1720 C828 P64758 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1188827 T1703 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1188826 T1734 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1487c491 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1188828 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 1188828 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 1188830 T1639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1188833 T1676 C827 P41573 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1188835 T1704 C826 P41239 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1188837 T1692 C829 P41374 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1188840 T1721 C828 P64758 oasc.SolrCore.execute [collection1] webapp=/_yfv/z path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1190846 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437694062925709312)} 0 2
[junit4:junit4] 2> 1190855 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={update.distrib=FROMLEADER&_version_=-1437694062929903616&update.from=http://127.0.0.1:41573/_yfv/z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437694062929903616)} 0 2
[junit4:junit4] 2> 1190859 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={update.distrib=FROMLEADER&_version_=-1437694062934097920&update.from=http://127.0.0.1:41374/_yfv/z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437694062934097920)} 0 2
[junit4:junit4] 2> 1190859 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437694062934097920)} 0 6
[junit4:junit4] 2> 1190860 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437694062929903616)} 0 11
[junit4:junit4] 2> 1190864 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[0 (1437694062944583680)]} 0 1
[junit4:junit4] 2> 1190871 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437694062948777984)]} 0 1
[junit4:junit4] 2> 1190872 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[0 (1437694062948777984)]} 0 5
[junit4:junit4] 2> 1190876 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[1 (1437694062957166592)]} 0 1
[junit4:junit4] 2> 1190883 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437694062961360896)]} 0 1
[junit4:junit4] 2> 1190883 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[1 (1437694062961360896)]} 0 4
[junit4:junit4] 2> 1190887 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[2 (1437694062968700928)]} 0 1
[junit4:junit4] 2> 1190893 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437694062971846656)]} 0 1
[junit4:junit4] 2> 1190894 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[2 (1437694062971846656)]} 0 5
[junit4:junit4] 2> 1190896 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[3 (1437694062979186688)]} 0 0
[junit4:junit4] 2> 1190906 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437694062985478144)]} 0 1
[junit4:junit4] 2> 1190906 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1437694062985478144)]} 0 4
[junit4:junit4] 2> 1190907 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[3]} 0 8
[junit4:junit4] 2> 1190911 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[4 (1437694062993866752)]} 0 1
[junit4:junit4] 2> 1190917 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437694062997012480)]} 0 1
[junit4:junit4] 2> 1190917 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[4 (1437694062997012480)]} 0 4
[junit4:junit4] 2> 1190921 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[5 (1437694063004352512)]} 0 1
[junit4:junit4] 2> 1190930 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437694063010643968)]} 0 1
[junit4:junit4] 2> 1190930 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1437694063010643968)]} 0 4
[junit4:junit4] 2> 1190931 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[5]} 0 8
[junit4:junit4] 2> 1190934 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[6 (1437694063019032576)]} 0 0
[junit4:junit4] 2> 1190941 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437694063022178304)]} 0 1
[junit4:junit4] 2> 1190941 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[6 (1437694063022178304)]} 0 4
[junit4:junit4] 2> 1190945 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[7 (1437694063029518336)]} 0 1
[junit4:junit4] 2> 1190953 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437694063034761216)]} 0 1
[junit4:junit4] 2> 1190953 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1437694063034761216)]} 0 4
[junit4:junit4] 2> 1190954 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[7]} 0 7
[junit4:junit4] 2> 1190958 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[8 (1437694063043149824)]} 0 1
[junit4:junit4] 2> 1190964 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437694063046295552)]} 0 1
[junit4:junit4] 2> 1190964 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[8 (1437694063046295552)]} 0 4
[junit4:junit4] 2> 1190967 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[9 (1437694063053635584)]} 0 0
[junit4:junit4] 2> 1190974 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437694063056781312)]} 0 1
[junit4:junit4] 2> 1190974 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[9 (1437694063056781312)]} 0 4
[junit4:junit4] 2> 1190978 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[10 (1437694063064121344)]} 0 1
[junit4:junit4] 2> 1190987 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437694063070412800)]} 0 1
[junit4:junit4] 2> 1190987 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1437694063070412800)]} 0 4
[junit4:junit4] 2> 1190988 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[10]} 0 8
[junit4:junit4] 2> 1190992 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[11 (1437694063078801408)]} 0 1
[junit4:junit4] 2> 1191001 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437694063085092864)]} 0 1
[junit4:junit4] 2> 1191001 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1437694063085092864)]} 0 4
[junit4:junit4] 2> 1191002 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[11]} 0 8
[junit4:junit4] 2> 1191006 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[12 (1437694063093481472)]} 0 1
[junit4:junit4] 2> 1191012 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437694063096627200)]} 0 1
[junit4:junit4] 2> 1191012 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[12 (1437694063096627200)]} 0 4
[junit4:junit4] 2> 1191016 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[13 (1437694063103967232)]} 0 1
[junit4:junit4] 2> 1191026 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437694063111307264)]} 0 1
[junit4:junit4] 2> 1191026 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1437694063111307264)]} 0 4
[junit4:junit4] 2> 1191027 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[13]} 0 8
[junit4:junit4] 2> 1191031 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[14 (1437694063119695872)]} 0 1
[junit4:junit4] 2> 1191040 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437694063125987328)]} 0 1
[junit4:junit4] 2> 1191040 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1437694063125987328)]} 0 4
[junit4:junit4] 2> 1191041 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[14]} 0 8
[junit4:junit4] 2> 1191045 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[15 (1437694063134375936)]} 0 1
[junit4:junit4] 2> 1191055 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437694063141715968)]} 0 1
[junit4:junit4] 2> 1191056 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1437694063141715968)]} 0 5
[junit4:junit4] 2> 1191056 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[15]} 0 8
[junit4:junit4] 2> 1191060 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[16 (1437694063150104576)]} 0 1
[junit4:junit4] 2> 1191070 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437694063157444608)]} 0 1
[junit4:junit4] 2> 1191071 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1437694063157444608)]} 0 5
[junit4:junit4] 2> 1191072 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[16]} 0 9
[junit4:junit4] 2> 1191076 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[17 (1437694063166881792)]} 0 1
[junit4:junit4] 2> 1191085 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437694063173173248)]} 0 1
[junit4:junit4] 2> 1191085 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1437694063173173248)]} 0 4
[junit4:junit4] 2> 1191086 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[17]} 0 8
[junit4:junit4] 2> 1191090 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[18 (1437694063181561856)]} 0 1
[junit4:junit4] 2> 1191099 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437694063187853312)]} 0 1
[junit4:junit4] 2> 1191099 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1437694063187853312)]} 0 4
[junit4:junit4] 2> 1191100 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[18]} 0 8
[junit4:junit4] 2> 1191104 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[19 (1437694063196241920)]} 0 1
[junit4:junit4] 2> 1191110 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437694063199387648)]} 0 1
[junit4:junit4] 2> 1191110 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[19 (1437694063199387648)]} 0 4
[junit4:junit4] 2> 1191113 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[20 (1437694063206727680)]} 0 0
[junit4:junit4] 2> 1191120 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437694063209873408)]} 0 1
[junit4:junit4] 2> 1191121 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[20 (1437694063209873408)]} 0 5
[junit4:junit4] 2> 1191124 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[21 (1437694063217213440)]} 0 1
[junit4:junit4] 2> 1191130 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437694063220359168)]} 0 1
[junit4:junit4] 2> 1191130 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[21 (1437694063220359168)]} 0 4
[junit4:junit4] 2> 1191134 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[22 (1437694063227699200)]} 0 1
[junit4:junit4] 2> 1191141 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437694063231893504)]} 0 1
[junit4:junit4] 2> 1191141 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[22 (1437694063231893504)]} 0 4
[junit4:junit4] 2> 1191145 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[23 (1437694063239233536)]} 0 1
[junit4:junit4] 2> 1191151 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437694063242379264)]} 0 1
[junit4:junit4] 2> 1191151 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[23 (1437694063242379264)]} 0 4
[junit4:junit4] 2> 1191154 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[24 (1437694063249719296)]} 0 0
[junit4:junit4] 2> 1191161 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437694063252865024)]} 0 1
[junit4:junit4] 2> 1191161 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[24 (1437694063252865024)]} 0 4
[junit4:junit4] 2> 1191165 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[25 (1437694063260205056)]} 0 1
[junit4:junit4] 2> 1191171 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437694063263350784)]} 0 1
[junit4:junit4] 2> 1191171 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[25 (1437694063263350784)]} 0 4
[junit4:junit4] 2> 1191175 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[26 (1437694063270690816)]} 0 1
[junit4:junit4] 2> 1191185 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437694063278030848)]} 0 0
[junit4:junit4] 2> 1191185 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1437694063278030848)]} 0 4
[junit4:junit4] 2> 1191186 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[26]} 0 8
[junit4:junit4] 2> 1191190 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[27 (1437694063286419456)]} 0 1
[junit4:junit4] 2> 1191199 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437694063292710912)]} 0 1
[junit4:junit4] 2> 1191199 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437694063292710912)]} 0 4
[junit4:junit4] 2> 1191200 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[27]} 0 8
[junit4:junit4] 2> 1191203 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[28 (1437694063301099520)]} 0 0
[junit4:junit4] 2> 1191210 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437694063304245248)]} 0 1
[junit4:junit4] 2> 1191210 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[28 (1437694063304245248)]} 0 4
[junit4:junit4] 2> 1191214 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[29 (1437694063311585280)]} 0 1
[junit4:junit4] 2> 1191220 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437694063314731008)]} 0 1
[junit4:junit4] 2> 1191220 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[29 (1437694063314731008)]} 0 4
[junit4:junit4] 2> 1191224 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[30 (1437694063322071040)]} 0 1
[junit4:junit4] 2> 1191230 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437694063325216768)]} 0 1
[junit4:junit4] 2> 1191230 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[30 (1437694063325216768)]} 0 4
[junit4:junit4] 2> 1191234 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[31 (1437694063332556800)]} 0 1
[junit4:junit4] 2> 1191243 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437694063338848256)]} 0 1
[junit4:junit4] 2> 1191243 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437694063338848256)]} 0 4
[junit4:junit4] 2> 1191244 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[31]} 0 8
[junit4:junit4] 2> 1191248 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[32 (1437694063347236864)]} 0 1
[junit4:junit4] 2> 1191254 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437694063350382592)]} 0 1
[junit4:junit4] 2> 1191254 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[32 (1437694063350382592)]} 0 4
[junit4:junit4] 2> 1191258 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[33 (1437694063357722624)]} 0 1
[junit4:junit4] 2> 1191264 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437694063360868352)]} 0 1
[junit4:junit4] 2> 1191264 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[33 (1437694063360868352)]} 0 4
[junit4:junit4] 2> 1191268 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[34 (1437694063368208384)]} 0 1
[junit4:junit4] 2> 1191277 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437694063374499840)]} 0 1
[junit4:junit4] 2> 1191277 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1437694063374499840)]} 0 4
[junit4:junit4] 2> 1191278 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[34]} 0 8
[junit4:junit4] 2> 1191282 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[35 (1437694063382888448)]} 0 1
[junit4:junit4] 2> 1191288 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437694063386034176)]} 0 1
[junit4:junit4] 2> 1191288 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[35 (1437694063386034176)]} 0 4
[junit4:junit4] 2> 1191292 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[36 (1437694063393374208)]} 0 1
[junit4:junit4] 2> 1191298 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437694063396519936)]} 0 1
[junit4:junit4] 2> 1191298 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[36 (1437694063396519936)]} 0 4
[junit4:junit4] 2> 1191302 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[37 (1437694063403859968)]} 0 1
[junit4:junit4] 2> 1191311 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437694063410151424)]} 0 1
[junit4:junit4] 2> 1191311 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1437694063410151424)]} 0 4
[junit4:junit4] 2> 1191312 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[37]} 0 8
[junit4:junit4] 2> 1191315 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[38 (1437694063418540032)]} 0 0
[junit4:junit4] 2> 1191322 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1437694063421685760)]} 0 1
[junit4:junit4] 2> 1191322 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[38 (1437694063421685760)]} 0 4
[junit4:junit4] 2> 1191326 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[39 (1437694063429025792)]} 0 1
[junit4:junit4] 2> 1191332 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1437694063432171520)]} 0 1
[junit4:junit4] 2> 1191332 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[39 (1437694063432171520)]} 0 4
[junit4:junit4] 2> 1191335 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[40 (1437694063439511552)]} 0 0
[junit4:junit4] 2> 1191345 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1437694063445803008)]} 0 1
[junit4:junit4] 2> 1191345 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1437694063445803008)]} 0 4
[junit4:junit4] 2> 1191346 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[40]} 0 8
[junit4:junit4] 2> 1191350 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[41 (1437694063454191616)]} 0 1
[junit4:junit4] 2> 1191358 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1437694063459434496)]} 0 1
[junit4:junit4] 2> 1191358 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1437694063459434496)]} 0 4
[junit4:junit4] 2> 1191359 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[41]} 0 7
[junit4:junit4] 2> 1191362 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[42 (1437694063467823104)]} 0 0
[junit4:junit4] 2> 1191372 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1437694063474114560)]} 0 1
[junit4:junit4] 2> 1191372 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1437694063474114560)]} 0 4
[junit4:junit4] 2> 1191373 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[42]} 0 8
[junit4:junit4] 2> 1191376 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[43 (1437694063482503168)]} 0 0
[junit4:junit4] 2> 1191386 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1437694063488794624)]} 0 1
[junit4:junit4] 2> 1191386 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1437694063488794624)]} 0 4
[junit4:junit4] 2> 1191387 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[43]} 0 8
[junit4:junit4] 2> 1191390 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[44 (1437694063497183232)]} 0 0
[junit4:junit4] 2> 1191400 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1437694063503474688)]} 0 1
[junit4:junit4] 2> 1191400 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1437694063503474688)]} 0 4
[junit4:junit4] 2> 1191401 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[44]} 0 8
[junit4:junit4] 2> 1191405 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[45 (1437694063511863296)]} 0 1
[junit4:junit4] 2> 1191411 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1437694063515009024)]} 0 1
[junit4:junit4] 2> 1191411 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[45 (1437694063515009024)]} 0 4
[junit4:junit4] 2> 1191415 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[46 (1437694063522349056)]} 0 0
[junit4:junit4] 2> 1191424 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1437694063528640512)]} 0 1
[junit4:junit4] 2> 1191424 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1437694063528640512)]} 0 4
[junit4:junit4] 2> 1191425 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[46]} 0 8
[junit4:junit4] 2> 1191429 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[47 (1437694063537029120)]} 0 1
[junit4:junit4] 2> 1191435 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1437694063540174848)]} 0 1
[junit4:junit4] 2> 1191435 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[47 (1437694063540174848)]} 0 4
[junit4:junit4] 2> 1191439 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[48 (1437694063547514880)]} 0 1
[junit4:junit4] 2> 1191448 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1437694063553806336)]} 0 1
[junit4:junit4] 2> 1191448 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1437694063553806336)]} 0 4
[junit4:junit4] 2> 1191449 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[48]} 0 8
[junit4:junit4] 2> 1191453 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[49 (1437694063562194944)]} 0 1
[junit4:junit4] 2> 1191459 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1437694063565340672)]} 0 0
[junit4:junit4] 2> 1191459 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[49 (1437694063565340672)]} 0 4
[junit4:junit4] 2> 1191463 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[50 (1437694063572680704)]} 0 1
[junit4:junit4] 2> 1191469 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1437694063575826432)]} 0 1
[junit4:junit4] 2> 1191469 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[50 (1437694063575826432)]} 0 4
[junit4:junit4] 2> 1191473 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[51 (1437694063583166464)]} 0 1
[junit4:junit4] 2> 1191482 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1437694063589457920)]} 0 1
[junit4:junit4] 2> 1191482 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1437694063589457920)]} 0 4
[junit4:junit4] 2> 1191483 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[51]} 0 8
[junit4:junit4] 2> 1191487 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[52 (1437694063597846528)]} 0 0
[junit4:junit4] 2> 1191493 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1437694063600992256)]} 0 1
[junit4:junit4] 2> 1191493 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[52 (1437694063600992256)]} 0 4
[junit4:junit4] 2> 1191497 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[53 (1437694063608332288)]} 0 1
[junit4:junit4] 2> 1191506 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1437694063614623744)]} 0 1
[junit4:junit4] 2> 1191506 T1693 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[53 (1437694063614623744)]} 0 4
[junit4:junit4] 2> 1191507 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[53]} 0 8
[junit4:junit4] 2> 1191511 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[54 (1437694063623012352)]} 0 1
[junit4:junit4] 2> 1191520 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1437694063629303808)]} 0 1
[junit4:junit4] 2> 1191520 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1437694063629303808)]} 0 4
[junit4:junit4] 2> 1191521 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[54]} 0 8
[junit4:junit4] 2> 1191525 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[55 (1437694063637692416)]} 0 1
[junit4:junit4] 2> 1191534 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1437694063643983872)]} 0 1
[junit4:junit4] 2> 1191534 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1437694063643983872)]} 0 4
[junit4:junit4] 2> 1191535 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[55]} 0 8
[junit4:junit4] 2> 1191539 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[56 (1437694063652372480)]} 0 1
[junit4:junit4] 2> 1191545 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1437694063655518208)]} 0 1
[junit4:junit4] 2> 1191545 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[56 (1437694063655518208)]} 0 4
[junit4:junit4] 2> 1191549 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[57 (1437694063662858240)]} 0 1
[junit4:junit4] 2> 1191555 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1437694063666003968)]} 0 1
[junit4:junit4] 2> 1191555 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[57 (1437694063666003968)]} 0 4
[junit4:junit4] 2> 1191559 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[58 (1437694063673344000)]} 0 1
[junit4:junit4] 2> 1191565 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1437694063676489728)]} 0 1
[junit4:junit4] 2> 1191565 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[58 (1437694063676489728)]} 0 4
[junit4:junit4] 2> 1191569 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[59 (1437694063683829760)]} 0 1
[junit4:junit4] 2> 1191578 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1437694063690121216)]} 0 1
[junit4:junit4] 2> 1191578 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1437694063690121216)]} 0 4
[junit4:junit4] 2> 1191579 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[59]} 0 8
[junit4:junit4] 2> 1191583 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[60 (1437694063698509824)]} 0 1
[junit4:junit4] 2> 1191589 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1437694063701655552)]} 0 1
[junit4:junit4] 2> 1191589 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[60 (1437694063701655552)]} 0 4
[junit4:junit4] 2> 1191593 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[61 (1437694063708995584)]} 0 1
[junit4:junit4] 2> 1191599 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1437694063712141312)]} 0 1
[junit4:junit4] 2> 1191599 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[61 (1437694063712141312)]} 0 4
[junit4:junit4] 2> 1191603 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[62 (1437694063719481344)]} 0 1
[junit4:junit4] 2> 1191609 T1720 C828 P64758 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1437694063722627072)]} 0 1
[junit4:junit4] 2> 1191609 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[62 (1437694063722627072)]} 0 4
[junit4:junit4] 2> 1191613 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[63 (1437694063729967104)]} 0 1
[junit4:junit4] 2> 1191622 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1437694063736258560)]} 0 1
[junit4:junit4] 2> 1191622 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41374/_yfv/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[63 (1437694063736258560)]} 0 4
[junit4:junit4] 2> 1191623 T1691 C829 P41374 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[63]} 0 8
[junit4:junit4] 2> 1191626 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[64 (1437694063744647168)]} 0 0
[junit4:junit4] 2> 1191633 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1437694063747792896)]} 0 1
[junit4:junit4] 2> 1191633 T1677 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[64 (1437694063747792896)]} 0 4
[junit4:junit4] 2> 1191636 T1651 C830 P21875 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={wt=javabin&version=2} {add=[65 (1437694063755132928)]} 0 0
[junit4:junit4] 2> 1191646 T1705 C826 P41239 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={distrib.from=http://127.0.0.1:41573/_yfv/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1437694063761424384)]} 0 1
[junit4:junit4] 2> 1191646 T1675 C827 P41573 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yfv/z path=/update params={dist
[...truncated too long message...]
erSync.sync PeerSync: core=collection1 url=http://127.0.0.1:64758/_yfv/z DONE. sync succeeded
[junit4:junit4] 2> 1463020 T1730 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 1463021 T1730 oasc.ShardLeaderElectionContext.runLeaderProcess Sync was not a success but no one else is active! I am the leader
[junit4:junit4] 2> 1463021 T1730 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64758/_yfv/z/collection1/
[junit4:junit4] 2> 1463021 T1730 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@24691a9f
[junit4:junit4] 2> 1463022 T1730 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=822,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1463023 T1730 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1463023 T1730 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1463023 T1730 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1463026 T1730 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1463027 T1730 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1463027 T1730 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index;done=false>>]
[junit4:junit4] 2> 1463027 T1730 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4/index
[junit4:junit4] 2> 1463027 T1730 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4;done=false>>]
[junit4:junit4] 2> 1463028 T1730 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371091874556/jetty4
[junit4:junit4] 2> 1463028 T1730 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1463028 T1730 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89855877091426316-127.0.0.1:64758__yfv%2Fz_collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 1463029 T1730 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1463029 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1463030 T1730 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1463030 T1730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1463030 T1730 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1463031 T1730 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=en_ZA, timezone=Atlantic/Madeira
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=429668240,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestCSVResponseWriter, TestUpdate, SOLR749Test, ZkControllerTest, TestDocSet, TestLMDirichletSimilarityFactory, TestMultiCoreConfBootstrap, ResponseLogComponentTest, TestSolrXml, IndexSchemaTest, TestFuzzyAnalyzedSuggestions, NumericFieldsTest, SolrRequestParserTest, ChaosMonkeyNothingIsSafeTest, TestSolrIndexConfig, SpellCheckCollatorTest, TestManagedSchemaFieldResource, TestDefaultSearchFieldResource, TestStressReorder, MoreLikeThisHandlerTest, TestConfig, AlternateDirectoryTest, TestSearchPerf, TestXIncludeConfig, TestPseudoReturnFields, UUIDFieldTest, FileUtilsTest, TestSystemIdResolver, TestSchemaNameResource, QueryParsingTest, CachingDirectoryFactoryTest, TestDynamicFieldResource, DocumentAnalysisRequestHandlerTest, DocValuesMultiTest, TestCSVLoader, QueryEqualityTest, PreAnalyzedUpdateProcessorTest, RequestHandlersTest, TestGroupingSearch, BadComponentTest, FullSolrCloudDistribCmdsTest, BasicDistributedZk2Test, TestAnalyzedSuggestions, PrimitiveFieldTypeTest, ResourceLoaderTest, TestRandomFaceting, TestBM25SimilarityFactory, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, TestCoreContainer, TestPluginEnable, PathHierarchyTokenizerFactoryTest, TestJoin, SynonymTokenizerTest, TestLFUCache, BasicFunctionalityTest, TestDynamicFieldCollectionResource, TestCollationKeyRangeQueries, TestStressRecovery, TestFastOutputStream, TestRandomDVFaceting, FieldMutatingUpdateProcessorTest, TestFieldResource, TestBinaryField, OpenExchangeRatesOrgProviderTest, TestSurroundQueryParser, SliceStateTest, TestStressVersions, TestQuerySenderNoQuery, SearchHandlerTest, TestStressLucene, TestCodecSupport, StandardRequestHandlerTest, TestAtomicUpdateErrorCases, BasicZkTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 300.95s, 1 test, 1 error <<< FAILURES!
[...truncated 449 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 1 error, 13 ignored (7 assumptions)
Total time: 54 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure