You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/05/27 07:35:05 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java6 - Build # 1658 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/1658/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:61917/p/gi returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:61917/p/gi returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([FD0639A356A19E1A:7CE0B7BB21FEFE26]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:679)
Build Log:
[...truncated 8890 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 404283 T1050 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /p/gi
[junit4:junit4] 2> 404287 T1050 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1369631160785
[junit4:junit4] 2> 404288 T1050 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 404289 T1051 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 404390 T1050 oasc.ZkTestServer.run start zk server on port:61910
[junit4:junit4] 2> 404390 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 404394 T1057 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f564967 name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 404394 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 404395 T1050 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 404405 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 404406 T1059 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@509c79d4 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 404406 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 404407 T1050 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 404415 T1050 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 404417 T1050 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 404425 T1050 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 404428 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 404428 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 404439 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 404440 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 404443 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 404444 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 404455 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 404456 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 404459 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 404459 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 404462 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 404463 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 404471 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 404472 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 404475 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 404479 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 404482 T1050 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 404483 T1050 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 404662 T1050 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 404666 T1050 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61913
[junit4:junit4] 2> 404667 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 404667 T1050 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 404667 T1050 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991
[junit4:junit4] 2> 404668 T1050 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/solr.xml
[junit4:junit4] 2> 404668 T1050 oasc.CoreContainer.<init> New CoreContainer 449804870
[junit4:junit4] 2> 404669 T1050 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/'
[junit4:junit4] 2> 404669 T1050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/'
[junit4:junit4] 2> 404743 T1050 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 404744 T1050 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 404744 T1050 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 404745 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 404745 T1050 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 404745 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 404746 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 404746 T1050 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 404746 T1050 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 404747 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 404757 T1050 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 404757 T1050 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61910/solr
[junit4:junit4] 2> 404758 T1050 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 404759 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 404763 T1070 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5910c667 name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 404764 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 404766 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 404772 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 404774 T1072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d118948 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 404774 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 404776 T1050 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 404786 T1050 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 404793 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 404795 T1050 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61913_p%2Fgi
[junit4:junit4] 2> 404796 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61913_p%2Fgi
[junit4:junit4] 2> 404806 T1050 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 404821 T1050 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 404824 T1050 oasc.Overseer.start Overseer (id=89760147753533443-127.0.0.1:61913_p%2Fgi-n_0000000000) starting
[junit4:junit4] 2> 404828 T1050 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 404841 T1074 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 404842 T1050 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 404844 T1050 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 404852 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 404854 T1073 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 404858 T1075 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/collection1
[junit4:junit4] 2> 404858 T1075 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 404859 T1075 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 404859 T1075 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 404860 T1075 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/collection1/'
[junit4:junit4] 2> 404861 T1075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/collection1/lib/README' to classloader
[junit4:junit4] 2> 404861 T1075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 404896 T1075 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 404934 T1075 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 404935 T1075 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 404939 T1075 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 405474 T1075 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 405478 T1075 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 405481 T1075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 405486 T1075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 405511 T1075 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 405511 T1075 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369631160991/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data/
[junit4:junit4] 2> 405512 T1075 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5fb11b79
[junit4:junit4] 2> 405512 T1075 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 405513 T1075 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data
[junit4:junit4] 2> 405513 T1075 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data/index/
[junit4:junit4] 2> 405514 T1075 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 405514 T1075 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data/index
[junit4:junit4] 2> 405522 T1075 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@643af383 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ff5784f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 405522 T1075 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 405534 T1075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 405534 T1075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 405535 T1075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 405536 T1075 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 405536 T1075 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 405537 T1075 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 405537 T1075 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 405538 T1075 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 405538 T1075 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 405548 T1075 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 405555 T1075 oass.SolrIndexSearcher.<init> Opening Searcher@5577f5b5 main
[junit4:junit4] 2> 405555 T1075 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/control/data/tlog
[junit4:junit4] 2> 405557 T1075 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 405557 T1075 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 405561 T1076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5577f5b5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 405563 T1075 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 405563 T1075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 406357 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 406358 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61913_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61913/p/gi"}
[junit4:junit4] 2> 406359 T1073 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 406359 T1073 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 406399 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 406584 T1075 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 406584 T1075 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:61913/p/gi collection:control_collection shard:shard1
[junit4:junit4] 2> 406585 T1075 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 406610 T1075 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 406621 T1075 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 406621 T1075 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 406622 T1075 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:61913/p/gi/collection1/
[junit4:junit4] 2> 406622 T1075 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 406622 T1075 oasc.SyncStrategy.syncToMe http://127.0.0.1:61913/p/gi/collection1/ has no replicas
[junit4:junit4] 2> 406623 T1075 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:61913/p/gi/collection1/
[junit4:junit4] 2> 406623 T1075 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 407903 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 407926 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 407955 T1075 oasc.ZkController.register We are http://127.0.0.1:61913/p/gi/collection1/ and leader is http://127.0.0.1:61913/p/gi/collection1/
[junit4:junit4] 2> 407955 T1075 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:61913/p/gi
[junit4:junit4] 2> 407955 T1075 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 407956 T1075 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 407956 T1075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 407958 T1075 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 407959 T1050 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4] 2> 407959 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 407960 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 407975 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 407977 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 407979 T1079 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fc2c671 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 407979 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 407981 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 407983 T1050 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 408238 T1050 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 408241 T1050 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61917
[junit4:junit4] 2> 408242 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 408242 T1050 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 408243 T1050 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481
[junit4:junit4] 2> 408244 T1050 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/solr.xml
[junit4:junit4] 2> 408244 T1050 oasc.CoreContainer.<init> New CoreContainer 1507158676
[junit4:junit4] 2> 408245 T1050 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/'
[junit4:junit4] 2> 408245 T1050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/'
[junit4:junit4] 2> 408355 T1050 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 408356 T1050 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 408356 T1050 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 408357 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 408358 T1050 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 408358 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 408359 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 408359 T1050 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 408360 T1050 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 408360 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 408377 T1050 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 408378 T1050 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61910/solr
[junit4:junit4] 2> 408378 T1050 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 408379 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 408383 T1090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f97e4d2 name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 408383 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 408398 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 408409 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 408411 T1092 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@830176e name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 408411 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 408422 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 409426 T1050 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61917_p%2Fgi
[junit4:junit4] 2> 409428 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61917_p%2Fgi
[junit4:junit4] 2> 409432 T1072 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 409432 T1092 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 409432 T1079 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 409432 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 409433 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 409441 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:61913_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61913_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61913/p/gi"}
[junit4:junit4] 2> 409451 T1093 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/collection1
[junit4:junit4] 2> 409452 T1093 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 409453 T1093 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 409453 T1093 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 409453 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 409453 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 409455 T1093 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/collection1/'
[junit4:junit4] 2> 409456 T1093 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/collection1/lib/README' to classloader
[junit4:junit4] 2> 409457 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 409457 T1093 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 409507 T1093 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 409564 T1093 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 409565 T1093 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 409571 T1093 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 410115 T1093 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 410119 T1093 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 410122 T1093 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 410127 T1093 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 410150 T1093 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 410151 T1093 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369631164481/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1/
[junit4:junit4] 2> 410151 T1093 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5fb11b79
[junit4:junit4] 2> 410152 T1093 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 410153 T1093 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1
[junit4:junit4] 2> 410153 T1093 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1/index/
[junit4:junit4] 2> 410153 T1093 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 410154 T1093 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1/index
[junit4:junit4] 2> 410158 T1093 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 410158 T1093 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 410161 T1093 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 410161 T1093 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 410162 T1093 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 410163 T1093 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 410164 T1093 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 410164 T1093 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 410164 T1093 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 410165 T1093 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 410166 T1093 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 410175 T1093 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 410182 T1093 oass.SolrIndexSearcher.<init> Opening Searcher@7df9088f main
[junit4:junit4] 2> 410183 T1093 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty1/tlog
[junit4:junit4] 2> 410184 T1093 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 410184 T1093 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 410189 T1094 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7df9088f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 410191 T1093 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 410192 T1093 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 410958 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 410959 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61917_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61917/p/gi"}
[junit4:junit4] 2> 410960 T1073 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 410960 T1073 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 411003 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 411003 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 411003 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 411194 T1093 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 411194 T1093 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:61917/p/gi collection:collection1 shard:shard1
[junit4:junit4] 2> 411195 T1093 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 411215 T1093 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 411226 T1093 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 411226 T1093 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 411226 T1093 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:61917/p/gi/collection1/
[junit4:junit4] 2> 411227 T1093 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 411227 T1093 oasc.SyncStrategy.syncToMe http://127.0.0.1:61917/p/gi/collection1/ has no replicas
[junit4:junit4] 2> 411227 T1093 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:61917/p/gi/collection1/
[junit4:junit4] 2> 411228 T1093 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 412507 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 412526 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 412526 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 412526 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 412560 T1093 oasc.ZkController.register We are http://127.0.0.1:61917/p/gi/collection1/ and leader is http://127.0.0.1:61917/p/gi/collection1/
[junit4:junit4] 2> 412560 T1093 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:61917/p/gi
[junit4:junit4] 2> 412560 T1093 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 412561 T1093 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 412561 T1093 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 412563 T1093 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 412564 T1050 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4] 2> 412565 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 412565 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 412840 T1050 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 412843 T1050 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57459
[junit4:junit4] 2> 412844 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 412845 T1050 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 412845 T1050 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078
[junit4:junit4] 2> 412846 T1050 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/solr.xml
[junit4:junit4] 2> 412846 T1050 oasc.CoreContainer.<init> New CoreContainer 1409281952
[junit4:junit4] 2> 412847 T1050 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/'
[junit4:junit4] 2> 412848 T1050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/'
[junit4:junit4] 2> 412959 T1050 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 412960 T1050 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 412960 T1050 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 412961 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 412962 T1050 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 412962 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 412963 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 412963 T1050 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 412964 T1050 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 412964 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 412980 T1050 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 412981 T1050 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61910/solr
[junit4:junit4] 2> 412981 T1050 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 412982 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 412986 T1106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b076a26 name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 412986 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 413003 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 413013 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 413015 T1108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3242af95 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 413015 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 413025 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 414028 T1050 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57459_p%2Fgi
[junit4:junit4] 2> 414030 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57459_p%2Fgi
[junit4:junit4] 2> 414033 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 414033 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 414033 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 414034 T1108 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 414035 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:61917_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61917_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61917/p/gi"}
[junit4:junit4] 2> 414035 T1079 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 414048 T1092 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 414052 T1109 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/collection1
[junit4:junit4] 2> 414053 T1109 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 414055 T1109 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 414055 T1109 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 414056 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 414056 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 414056 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 414058 T1109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/collection1/'
[junit4:junit4] 2> 414059 T1109 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 414059 T1072 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 414060 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 414060 T1109 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/collection1/lib/README' to classloader
[junit4:junit4] 2> 414061 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 414110 T1109 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 414167 T1109 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 414168 T1109 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 414174 T1109 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 414719 T1109 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 414723 T1109 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 414726 T1109 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 414731 T1109 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 414754 T1109 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 414755 T1109 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369631169078/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2/
[junit4:junit4] 2> 414755 T1109 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5fb11b79
[junit4:junit4] 2> 414756 T1109 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 414756 T1109 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2
[junit4:junit4] 2> 414757 T1109 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2/index/
[junit4:junit4] 2> 414757 T1109 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 414758 T1109 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2/index
[junit4:junit4] 2> 414762 T1109 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_1,generation=1}
[junit4:junit4] 2> 414762 T1109 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 414765 T1109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 414766 T1109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 414767 T1109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 414767 T1109 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 414768 T1109 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 414768 T1109 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 414769 T1109 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 414770 T1109 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 414770 T1109 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 414780 T1109 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 414788 T1109 oass.SolrIndexSearcher.<init> Opening Searcher@71a7c9f8 main
[junit4:junit4] 2> 414788 T1109 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty2/tlog
[junit4:junit4] 2> 414789 T1109 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 414790 T1109 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 414795 T1110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71a7c9f8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 414797 T1109 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 414798 T1109 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 415562 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 415563 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57459_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57459/p/gi"}
[junit4:junit4] 2> 415563 T1073 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 415563 T1073 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 415585 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 415585 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 415585 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 415585 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 415800 T1109 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 415800 T1109 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57459/p/gi collection:collection1 shard:shard2
[junit4:junit4] 2> 415801 T1109 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 415820 T1109 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 415830 T1109 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 415830 T1109 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 415830 T1109 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57459/p/gi/collection1/
[junit4:junit4] 2> 415831 T1109 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 415831 T1109 oasc.SyncStrategy.syncToMe http://127.0.0.1:57459/p/gi/collection1/ has no replicas
[junit4:junit4] 2> 415831 T1109 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57459/p/gi/collection1/
[junit4:junit4] 2> 415832 T1109 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 417089 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 417108 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 417109 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 417109 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 417109 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 417122 T1109 oasc.ZkController.register We are http://127.0.0.1:57459/p/gi/collection1/ and leader is http://127.0.0.1:57459/p/gi/collection1/
[junit4:junit4] 2> 417122 T1109 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57459/p/gi
[junit4:junit4] 2> 417122 T1109 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 417123 T1109 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 417123 T1109 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 417125 T1109 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 417126 T1050 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4] 2> 417127 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 417127 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 417401 T1050 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 417405 T1050 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50621
[junit4:junit4] 2> 417406 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 417406 T1050 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 417407 T1050 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640
[junit4:junit4] 2> 417407 T1050 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/solr.xml
[junit4:junit4] 2> 417408 T1050 oasc.CoreContainer.<init> New CoreContainer 458582041
[junit4:junit4] 2> 417409 T1050 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/'
[junit4:junit4] 2> 417409 T1050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/'
[junit4:junit4] 2> 417526 T1050 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 417527 T1050 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 417527 T1050 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 417528 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 417529 T1050 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 417529 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 417530 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 417530 T1050 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 417531 T1050 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 417531 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 417547 T1050 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 417547 T1050 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61910/solr
[junit4:junit4] 2> 417548 T1050 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 417549 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 417552 T1122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3493dd81 name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 417553 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 417567 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 417577 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 417579 T1124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61443ce2 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 417579 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 417589 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 418592 T1050 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50621_p%2Fgi
[junit4:junit4] 2> 418594 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50621_p%2Fgi
[junit4:junit4] 2> 418597 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 418598 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 418597 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 418598 T1124 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 418598 T1072 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 418599 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418599 T1079 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 418607 T1092 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 418607 T1108 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 418612 T1125 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/collection1
[junit4:junit4] 2> 418612 T1125 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 418613 T1125 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 418613 T1125 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 418614 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 418615 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/collection1/'
[junit4:junit4] 2> 418615 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57459_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57459_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57459/p/gi"}
[junit4:junit4] 2> 418616 T1125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/collection1/lib/README' to classloader
[junit4:junit4] 2> 418617 T1125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 418627 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418628 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418628 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418628 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418627 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 418667 T1125 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 418724 T1125 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 418726 T1125 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 418731 T1125 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 419276 T1125 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 419281 T1125 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 419283 T1125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 419288 T1125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 419312 T1125 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 419312 T1125 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369631173640/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/
[junit4:junit4] 2> 419313 T1125 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5fb11b79
[junit4:junit4] 2> 419314 T1125 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 419314 T1125 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3
[junit4:junit4] 2> 419315 T1125 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index/
[junit4:junit4] 2> 419315 T1125 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 419316 T1125 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index
[junit4:junit4] 2> 419320 T1125 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fb3e358 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c5c1eea),segFN=segments_1,generation=1}
[junit4:junit4] 2> 419320 T1125 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 419323 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 419323 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 419324 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 419325 T1125 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 419326 T1125 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 419326 T1125 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 419326 T1125 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 419327 T1125 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 419328 T1125 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 419338 T1125 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 419345 T1125 oass.SolrIndexSearcher.<init> Opening Searcher@42e88383 main
[junit4:junit4] 2> 419346 T1125 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/tlog
[junit4:junit4] 2> 419347 T1125 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 419347 T1125 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 419352 T1126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@42e88383 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 419354 T1125 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 419354 T1125 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 420132 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 420133 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50621_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50621/p/gi"}
[junit4:junit4] 2> 420134 T1073 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 420134 T1073 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 420148 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 420148 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 420148 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 420148 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 420148 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 420357 T1125 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 420357 T1125 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50621/p/gi collection:collection1 shard:shard1
[junit4:junit4] 2> 420364 T1125 oasc.ZkController.register We are http://127.0.0.1:50621/p/gi/collection1/ and leader is http://127.0.0.1:61917/p/gi/collection1/
[junit4:junit4] 2> 420365 T1125 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50621/p/gi
[junit4:junit4] 2> 420365 T1125 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 420365 T1125 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C146 name=collection1 org.apache.solr.core.SolrCore@23823385 url=http://127.0.0.1:50621/p/gi/collection1 node=127.0.0.1:50621_p%2Fgi C146_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50621_p%2Fgi, base_url=http://127.0.0.1:50621/p/gi}
[junit4:junit4] 2> 420366 T1127 C146 P50621 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 420366 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 420367 T1127 C146 P50621 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 420368 T1127 C146 P50621 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 420368 T1127 C146 P50621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 420368 T1050 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4] 2> 420369 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 420369 T1127 C146 P50621 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 420370 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 420394 T1082 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50621_p%2Fgi_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 420675 T1050 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 420678 T1050 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:55312
[junit4:junit4] 2> 420679 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 420679 T1050 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 420680 T1050 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889
[junit4:junit4] 2> 420680 T1050 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/solr.xml
[junit4:junit4] 2> 420681 T1050 oasc.CoreContainer.<init> New CoreContainer 1899982469
[junit4:junit4] 2> 420682 T1050 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/'
[junit4:junit4] 2> 420682 T1050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/'
[junit4:junit4] 2> 420797 T1050 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 420798 T1050 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 420798 T1050 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 420799 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 420799 T1050 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 420800 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 420800 T1050 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 420801 T1050 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 420802 T1050 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 420802 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 420819 T1050 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 420820 T1050 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61910/solr
[junit4:junit4] 2> 420820 T1050 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 420821 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 420824 T1139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78da29ee name:ZooKeeperConnection Watcher:127.0.0.1:61910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 420825 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 420842 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 420854 T1050 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 420856 T1141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a2434e7 name:ZooKeeperConnection Watcher:127.0.0.1:61910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 420856 T1050 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 420861 T1050 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 421652 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 421653 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50621_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50621_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50621/p/gi"}
[junit4:junit4] 2> 421671 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421671 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421671 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421671 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421671 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421671 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421865 T1050 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55312_p%2Fgi
[junit4:junit4] 2> 421867 T1050 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55312_p%2Fgi
[junit4:junit4] 2> 421869 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421869 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421869 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 421870 T1072 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421870 T1124 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421870 T1141 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421871 T1108 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421871 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 421871 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 421871 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 421878 T1092 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421878 T1079 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 421883 T1142 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/collection1
[junit4:junit4] 2> 421884 T1142 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 421884 T1142 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 421885 T1142 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 421886 T1142 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/collection1/'
[junit4:junit4] 2> 421888 T1142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 421888 T1142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/collection1/lib/README' to classloader
[junit4:junit4] 2> 421938 T1142 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 421995 T1142 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 421997 T1142 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 422002 T1142 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 422396 T1082 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50621_p%2Fgi_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 422396 T1082 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50621_p%252Fgi_collection1&state=recovering&nodeName=127.0.0.1:50621_p%252Fgi&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 422550 T1142 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 422554 T1142 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 422556 T1142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 422562 T1142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 422586 T1142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 422587 T1142 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369631176889/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/
[junit4:junit4] 2> 422587 T1142 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5fb11b79
[junit4:junit4] 2> 422588 T1142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 422588 T1142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4
[junit4:junit4] 2> 422589 T1142 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index/
[junit4:junit4] 2> 422589 T1142 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 422590 T1142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index
[junit4:junit4] 2> 422594 T1142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3edaa718 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5141f225),segFN=segments_1,generation=1}
[junit4:junit4] 2> 422594 T1142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 422597 T1142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 422598 T1142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 422598 T1142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 422599 T1142 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 422600 T1142 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 422600 T1142 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 422601 T1142 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 422601 T1142 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 422602 T1142 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 422611 T1142 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 422619 T1142 oass.SolrIndexSearcher.<init> Opening Searcher@2d0e0f04 main
[junit4:junit4] 2> 422619 T1142 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/tlog
[junit4:junit4] 2> 422620 T1142 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 422621 T1142 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 422626 T1143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d0e0f04 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 422628 T1142 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 422629 T1142 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 423176 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 423177 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55312_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55312/p/gi"}
[junit4:junit4] 2> 423177 T1073 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 423178 T1073 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 423201 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423201 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423201 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423201 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423201 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423201 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 423656 T1142 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 423656 T1142 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55312/p/gi collection:collection1 shard:shard2
[junit4:junit4] 2> 423659 T1142 oasc.ZkController.register We are http://127.0.0.1:55312/p/gi/collection1/ and leader is http://127.0.0.1:57459/p/gi/collection1/
[junit4:junit4] 2> 423660 T1142 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55312/p/gi
[junit4:junit4] 2> 423660 T1142 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 423660 T1142 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C147 name=collection1 org.apache.solr.core.SolrCore@11b2a12e url=http://127.0.0.1:55312/p/gi/collection1 node=127.0.0.1:55312_p%2Fgi C147_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:55312_p%2Fgi, base_url=http://127.0.0.1:55312/p/gi}
[junit4:junit4] 2> 423661 T1144 C147 P55312 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 423661 T1142 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 423662 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 423663 T1144 C147 P55312 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 423663 T1144 C147 P55312 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 423663 T1050 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4] 2> 423664 T1050 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 423664 T1144 C147 P55312 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 423664 T1050 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 423686 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 423687 T1098 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:55312_p%2Fgi_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 423688 T1050 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 423689 T1050 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 423689 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C148 name=collection1 org.apache.solr.core.SolrCore@23823385 url=http://127.0.0.1:50621/p/gi/collection1 node=127.0.0.1:50621_p%2Fgi C148_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50621_p%2Fgi, base_url=http://127.0.0.1:50621/p/gi}
[junit4:junit4] 2> 424398 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:61917/p/gi/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 424398 T1127 C148 P50621 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50621/p/gi START replicas=[http://127.0.0.1:61917/p/gi/collection1/] nUpdates=100
[junit4:junit4] 2> 424399 T1127 C148 P50621 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 424399 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 424400 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 424400 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 424400 T1127 C148 P50621 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 424401 T1127 C148 P50621 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:61917/p/gi/collection1/. core=collection1
[junit4:junit4] 2> 424401 T1127 C148 P50621 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C149 name=collection1 org.apache.solr.core.SolrCore@6a22c2c3 url=http://127.0.0.1:61917/p/gi/collection1 node=127.0.0.1:61917_p%2Fgi C149_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:61917_p%2Fgi, base_url=http://127.0.0.1:61917/p/gi, leader=true}
[junit4:junit4] 2> 424407 T1083 C149 P61917 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 424411 T1084 C149 P61917 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 424414 T1084 C149 P61917 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 424414 T1084 C149 P61917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 424415 T1084 C149 P61917 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_2,generation=2}
[junit4:junit4] 2> 424416 T1084 C149 P61917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 424416 T1084 C149 P61917 oass.SolrIndexSearcher.<init> Opening Searcher@584ced04 realtime
[junit4:junit4] 2> 424417 T1084 C149 P61917 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 424417 T1084 C149 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 424418 T1127 C148 P50621 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 424419 T1127 C148 P50621 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 424421 T1085 C149 P61917 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 424421 T1085 C149 P61917 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 424422 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 424422 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 424422 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 424425 T1085 C149 P61917 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 424426 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 424427 T1127 C148 P50621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index.20130527140620923
[junit4:junit4] 2> 424427 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@f5c2181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a6d35ef) fullCopy=false
[junit4:junit4] 2> 424430 T1085 C149 P61917 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 424431 T1127 C148 P50621 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 424432 T1127 C148 P50621 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 424432 T1127 C148 P50621 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 424434 T1127 C148 P50621 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fb3e358 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c5c1eea),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fb3e358 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c5c1eea),segFN=segments_2,generation=2}
[junit4:junit4] 2> 424435 T1127 C148 P50621 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 424436 T1127 C148 P50621 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 424436 T1127 C148 P50621 oass.SolrIndexSearcher.<init> Opening Searcher@20f3c7ad main
[junit4:junit4] 2> 424437 T1126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20f3c7ad main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 424438 T1127 C148 P50621 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index.20130527140620923 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index.20130527140620923;done=true>>]
[junit4:junit4] 2> 424438 T1127 C148 P50621 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index.20130527140620923
[junit4:junit4] 2> 424439 T1127 C148 P50621 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty3/index.20130527140620923
[junit4:junit4] 2> 424439 T1127 C148 P50621 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 424439 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 424439 T1127 C148 P50621 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 424440 T1127 C148 P50621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 424441 T1127 C148 P50621 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 424691 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 424705 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 424706 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55312_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55312_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55312/p/gi"}
[junit4:junit4] 2> 424746 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50621_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50621_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50621/p/gi"}
[junit4:junit4] 2> 424760 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 424760 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 424760 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 424760 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 424760 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 424760 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 425690 T1098 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:55312_p%2Fgi_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 425690 T1098 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:55312_p%252Fgi_collection1&state=recovering&nodeName=127.0.0.1:55312_p%252Fgi&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 425693 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 426695 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C147_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55312_p%2Fgi, base_url=http://127.0.0.1:55312/p/gi}
[junit4:junit4] 2> 427692 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57459/p/gi/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 427692 T1144 C147 P55312 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55312/p/gi START replicas=[http://127.0.0.1:57459/p/gi/collection1/] nUpdates=100
[junit4:junit4] 2> 427693 T1144 C147 P55312 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 427694 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 427694 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 427694 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 427694 T1144 C147 P55312 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 427695 T1144 C147 P55312 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57459/p/gi/collection1/. core=collection1
[junit4:junit4] 2> 427696 T1144 C147 P55312 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 427698 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C150 name=collection1 org.apache.solr.core.SolrCore@59f69748 url=http://127.0.0.1:57459/p/gi/collection1 node=127.0.0.1:57459_p%2Fgi C150_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57459_p%2Fgi, base_url=http://127.0.0.1:57459/p/gi, leader=true}
[junit4:junit4] 2> 427702 T1099 C150 P57459 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 427707 T1100 C150 P57459 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 427709 T1100 C150 P57459 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_1,generation=1}
[junit4:junit4] 2> 427710 T1100 C150 P57459 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 427711 T1100 C150 P57459 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_2,generation=2}
[junit4:junit4] 2> 427711 T1100 C150 P57459 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 427712 T1100 C150 P57459 oass.SolrIndexSearcher.<init> Opening Searcher@17b791 realtime
[junit4:junit4] 2> 427712 T1100 C150 P57459 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 427713 T1100 C150 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 427714 T1144 C147 P55312 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 427714 T1144 C147 P55312 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 427716 T1101 C150 P57459 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 427716 T1101 C150 P57459 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 427717 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 427717 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 427717 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 427720 T1101 C150 P57459 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 427721 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 427722 T1144 C147 P55312 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index.20130527140624218
[junit4:junit4] 2> 427722 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@2711815d lockFactory=org.apache.lucene.store.NativeFSLockFactory@57c154a3) fullCopy=false
[junit4:junit4] 2> 427725 T1101 C150 P57459 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 427726 T1144 C147 P55312 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 427727 T1144 C147 P55312 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 427727 T1144 C147 P55312 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 427729 T1144 C147 P55312 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3edaa718 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5141f225),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3edaa718 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5141f225),segFN=segments_2,generation=2}
[junit4:junit4] 2> 427730 T1144 C147 P55312 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 427730 T1144 C147 P55312 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 427730 T1144 C147 P55312 oass.SolrIndexSearcher.<init> Opening Searcher@1fe0d56a main
[junit4:junit4] 2> 427732 T1143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fe0d56a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 427732 T1144 C147 P55312 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index.20130527140624218 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index.20130527140624218;done=true>>]
[junit4:junit4] 2> 427733 T1144 C147 P55312 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index.20130527140624218
[junit4:junit4] 2> 427733 T1144 C147 P55312 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index.20130527140624218
[junit4:junit4] 2> 427733 T1144 C147 P55312 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 427733 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 427734 T1144 C147 P55312 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 427734 T1144 C147 P55312 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 427736 T1144 C147 P55312 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 427776 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 427777 T1073 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55312_p%2Fgi_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55312_p%2Fgi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55312/p/gi"}
[junit4:junit4] 2> 427789 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 427789 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 427789 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 427789 T1079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 427789 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 427789 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 428700 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 428701 T1050 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C151 name=collection1 org.apache.solr.core.SolrCore@79e893ae url=http://127.0.0.1:61913/p/gi/collection1 node=127.0.0.1:61913_p%2Fgi C151_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:61913_p%2Fgi, base_url=http://127.0.0.1:61913/p/gi, leader=true}
[junit4:junit4] 2> 428710 T1062 C151 P61913 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 428712 T1062 C151 P61913 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@643af383 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ff5784f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 428713 T1062 C151 P61913 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 428714 T1062 C151 P61913 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@643af383 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ff5784f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@643af383 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ff5784f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 428715 T1062 C151 P61913 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 428716 T1062 C151 P61913 oass.SolrIndexSearcher.<init> Opening Searcher@b929312 main
[junit4:junit4] 2> 428716 T1062 C151 P61913 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 428717 T1076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b929312 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 428718 T1062 C151 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 428730 T1086 C149 P61917 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 428731 T1086 C149 P61917 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a29bb76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@449ff1d3),segFN=segments_3,generation=3}
[junit4:junit4] 2> 428732 T1086 C149 P61917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 428732 T1086 C149 P61917 oass.SolrIndexSearcher.<init> Opening Searcher@1e4f430f main
[junit4:junit4] 2> 428733 T1086 C149 P61917 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 428734 T1094 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e4f430f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 428734 T1086 C149 P61917 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50621/p/gi/collection1/, StdNode: http://127.0.0.1:57459/p/gi/collection1/, StdNode: http://127.0.0.1:55312/p/gi/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 428737 T1102 C150 P57459 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 428739 T1102 C150 P57459 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@589d0bde lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fbf2845),segFN=segments_3,generation=3}
[junit4:junit4] 2> 428739 T1102 C150 P57459 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 428740 T1102 C150 P57459 oass.SolrIndexSearcher.<init> Opening Searcher@2f59f0d1 main
[junit4:junit4] 2> 428740 T1102 C150 P57459 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 428741 T1110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f59f0d1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 428742 T1102 C150 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> ASYNC NEW_CORE C152 name=collection1 org.apache.solr.core.SolrCore@23823385 url=http://127.0.0.1:50621/p/gi/collection1 node=127.0.0.1:50621_p%2Fgi C152_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50621_p%2Fgi, base_url=http://127.0.0.1:50621/p/gi}
[junit4:junit4] 2> 428750 T1114 C152 P50621 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C153 name=collection1 org.apache.solr.core.SolrCore@11b2a12e url=http://127.0.0.1:55312/p/gi/collection1 node=127.0.0.1:55312_p%2Fgi C153_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55312_p%2Fgi, base_url=http://127.0.0.1:55312/p/gi}
[junit4:junit4] 2> 428750 T1131 C153 P55312 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 428751 T1114 C152 P50621 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fb3e358 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c5c1eea),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fb3e358 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c5c1eea),segFN=segments_3,generation=3}
[junit4:junit4] 2> 428752 T1131 C153 P55312 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3edaa718 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5141f225),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3edaa718 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5141f225),segFN=segments_3,generation=3}
[junit4:junit4] 2> 428752 T1114 C152 P50621 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 428753 T1131 C153 P55312 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 428753 T1114 C152 P50621 oass.SolrIndexSearcher.<init> Opening Searcher@968605 main
[junit4:junit4] 2> 428754 T1131 C153 P55312 oass.SolrIndexSearcher.<init> Opening Searcher@531963 main
[junit4:junit4] 2> 428754 T1114 C152 P50621 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 428755 T1131 C153 P55312 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 428755 T1126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@968605 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 428756 T1143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@531963 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 428756 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 428757 T1131 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> ASYNC NEW_CORE C154 name=collection1 org.apache.solr.core.SolrCore@6a22c2c3 url=http://127.0.0.1:61917/p/gi/collection1 node=127.0.0.1:61917_p%2Fgi C154_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:61917_p%2Fgi, base_url=http://127.0.0.1:61917/p/gi, leader=true}
[junit4:junit4] 2> 428758 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4] 2> 428759 T1050 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 428762 T1087 C154 P61917 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 428764 T1115 C152 P50621 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C155 name=collection1 org.apache.solr.core.SolrCore@59f69748 url=http://127.0.0.1:57459/p/gi/collection1 node=127.0.0.1:57459_p%2Fgi C155_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57459_p%2Fgi, base_url=http://127.0.0.1:57459/p/gi, leader=true}
[junit4:junit4] 2> 428766 T1103 C155 P57459 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 428769 T1132 C153 P55312 oasc.SolrCore.execute [collection1] webapp=/p/gi path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> ASYNC NEW_CORE C156 name=collection1 org.apache.solr.core.SolrCore@79e893ae url=http://127.0.0.1:61913/p/gi/collection1 node=127.0.0.1:61913_p%2Fgi C156_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:61913_p%2Fgi, base_url=http://127.0.0.1:61913/p/gi, leader=true}
[junit4:junit4] 2> 430796 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436162391822827520)} 0 24
[junit4:junit4] 2> 430804 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={update.distrib=FROMLEADER&_version_=-1436162391850090496&update.from=http://127.0.0.1:61917/p/gi/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436162391850090496)} 0 1
[junit4:junit4] 2> 430808 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={update.distrib=FROMLEADER&_version_=-1436162391854284800&update.from=http://127.0.0.1:57459/p/gi/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436162391854284800)} 0 1
[junit4:junit4] 2> 430809 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1436162391854284800)} 0 6
[junit4:junit4] 2> 430810 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436162391850090496)} 0 11
[junit4:junit4] 2> 430814 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[0 (1436162391864770560)]} 0 1
[junit4:junit4] 2> 430821 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1436162391868964864)]} 0 1
[junit4:junit4] 2> 430822 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1436162391868964864)]} 0 5
[junit4:junit4] 2> 430822 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[0]} 0 6
[junit4:junit4] 2> 430825 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[1 (1436162391877353472)]} 0 0
[junit4:junit4] 2> 430831 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1436162391880499200)]} 0 0
[junit4:junit4] 2> 430832 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[1 (1436162391880499200)]} 0 4
[junit4:junit4] 2> 430836 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[2 (1436162391887839232)]} 0 1
[junit4:junit4] 2> 430842 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1436162391890984960)]} 0 1
[junit4:junit4] 2> 430842 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[2 (1436162391890984960)]} 0 4
[junit4:junit4] 2> 430845 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[3 (1436162391898324992)]} 0 0
[junit4:junit4] 2> 430853 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1436162391903567872)]} 0 0
[junit4:junit4] 2> 430854 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1436162391903567872)]} 0 4
[junit4:junit4] 2> 430855 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4] 2> 430858 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[4 (1436162391911956480)]} 0 0
[junit4:junit4] 2> 430864 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1436162391915102208)]} 0 0
[junit4:junit4] 2> 430865 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[4 (1436162391915102208)]} 0 4
[junit4:junit4] 2> 430868 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[5 (1436162391922442240)]} 0 0
[junit4:junit4] 2> 430874 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1436162391925587968)]} 0 0
[junit4:junit4] 2> 430875 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[5 (1436162391925587968)]} 0 4
[junit4:junit4] 2> 430878 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[6 (1436162391932928000)]} 0 0
[junit4:junit4] 2> 430884 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1436162391936073728)]} 0 0
[junit4:junit4] 2> 430885 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[6 (1436162391936073728)]} 0 4
[junit4:junit4] 2> 430888 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[7 (1436162391943413760)]} 0 0
[junit4:junit4] 2> 430894 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1436162391946559488)]} 0 0
[junit4:junit4] 2> 430895 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[7 (1436162391946559488)]} 0 4
[junit4:junit4] 2> 430898 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[8 (1436162391953899520)]} 0 0
[junit4:junit4] 2> 430904 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1436162391957045248)]} 0 0
[junit4:junit4] 2> 430905 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[8 (1436162391957045248)]} 0 4
[junit4:junit4] 2> 430908 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[9 (1436162391964385280)]} 0 0
[junit4:junit4] 2> 430916 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1436162391969628160)]} 0 0
[junit4:junit4] 2> 430917 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1436162391969628160)]} 0 4
[junit4:junit4] 2> 430918 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[9]} 0 7
[junit4:junit4] 2> 430921 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[10 (1436162391978016768)]} 0 0
[junit4:junit4] 2> 430927 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1436162391981162496)]} 0 0
[junit4:junit4] 2> 430928 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[10 (1436162391981162496)]} 0 4
[junit4:junit4] 2> 430931 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[11 (1436162391988502528)]} 0 0
[junit4:junit4] 2> 430938 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1436162391991648256)]} 0 1
[junit4:junit4] 2> 430938 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[11 (1436162391991648256)]} 0 4
[junit4:junit4] 2> 430941 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[12 (1436162391998988288)]} 0 0
[junit4:junit4] 2> 430947 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1436162392002134016)]} 0 0
[junit4:junit4] 2> 430948 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[12 (1436162392002134016)]} 0 4
[junit4:junit4] 2> 430951 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[13 (1436162392009474048)]} 0 0
[junit4:junit4] 2> 430957 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1436162392012619776)]} 0 0
[junit4:junit4] 2> 430958 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[13 (1436162392012619776)]} 0 4
[junit4:junit4] 2> 430961 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[14 (1436162392019959808)]} 0 0
[junit4:junit4] 2> 430969 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1436162392025202688)]} 0 0
[junit4:junit4] 2> 430970 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1436162392025202688)]} 0 4
[junit4:junit4] 2> 430971 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[14]} 0 7
[junit4:junit4] 2> 430974 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[15 (1436162392033591296)]} 0 0
[junit4:junit4] 2> 430982 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1436162392038834176)]} 0 0
[junit4:junit4] 2> 430983 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1436162392038834176)]} 0 4
[junit4:junit4] 2> 430984 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 430987 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[16 (1436162392047222784)]} 0 0
[junit4:junit4] 2> 430995 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1436162392052465664)]} 0 0
[junit4:junit4] 2> 430996 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1436162392052465664)]} 0 4
[junit4:junit4] 2> 430997 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[16]} 0 7
[junit4:junit4] 2> 431000 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[17 (1436162392060854272)]} 0 0
[junit4:junit4] 2> 431009 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1436162392066097152)]} 0 1
[junit4:junit4] 2> 431009 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1436162392066097152)]} 0 4
[junit4:junit4] 2> 431010 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4] 2> 431013 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[18 (1436162392074485760)]} 0 0
[junit4:junit4] 2> 431019 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1436162392077631488)]} 0 0
[junit4:junit4] 2> 431020 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[18 (1436162392077631488)]} 0 4
[junit4:junit4] 2> 431023 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[19 (1436162392084971520)]} 0 0
[junit4:junit4] 2> 431029 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1436162392088117248)]} 0 0
[junit4:junit4] 2> 431030 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[19 (1436162392088117248)]} 0 4
[junit4:junit4] 2> 431033 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[20 (1436162392095457280)]} 0 0
[junit4:junit4] 2> 431039 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1436162392098603008)]} 0 0
[junit4:junit4] 2> 431040 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[20 (1436162392098603008)]} 0 4
[junit4:junit4] 2> 431043 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[21 (1436162392105943040)]} 0 0
[junit4:junit4] 2> 431051 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1436162392111185920)]} 0 0
[junit4:junit4] 2> 431052 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1436162392111185920)]} 0 4
[junit4:junit4] 2> 431053 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[21]} 0 7
[junit4:junit4] 2> 431056 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[22 (1436162392119574528)]} 0 0
[junit4:junit4] 2> 431062 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1436162392122720256)]} 0 0
[junit4:junit4] 2> 431063 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[22 (1436162392122720256)]} 0 4
[junit4:junit4] 2> 431066 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[23 (1436162392130060288)]} 0 0
[junit4:junit4] 2> 431074 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1436162392135303168)]} 0 0
[junit4:junit4] 2> 431075 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1436162392135303168)]} 0 4
[junit4:junit4] 2> 431076 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[23]} 0 7
[junit4:junit4] 2> 431080 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[24 (1436162392143691776)]} 0 0
[junit4:junit4] 2> 431085 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1436162392146837504)]} 0 0
[junit4:junit4] 2> 431086 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[24 (1436162392146837504)]} 0 4
[junit4:junit4] 2> 431089 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[25 (1436162392154177536)]} 0 0
[junit4:junit4] 2> 431097 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1436162392159420416)]} 0 0
[junit4:junit4] 2> 431098 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1436162392159420416)]} 0 4
[junit4:junit4] 2> 431099 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4] 2> 431102 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[26 (1436162392167809024)]} 0 0
[junit4:junit4] 2> 431111 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1436162392173051904)]} 0 1
[junit4:junit4] 2> 431111 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1436162392173051904)]} 0 4
[junit4:junit4] 2> 431112 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[26]} 0 7
[junit4:junit4] 2> 431115 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[27 (1436162392181440512)]} 0 0
[junit4:junit4] 2> 431121 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1436162392184586240)]} 0 0
[junit4:junit4] 2> 431122 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[27 (1436162392184586240)]} 0 4
[junit4:junit4] 2> 431125 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[28 (1436162392191926272)]} 0 0
[junit4:junit4] 2> 431133 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1436162392197169152)]} 0 0
[junit4:junit4] 2> 431134 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1436162392197169152)]} 0 4
[junit4:junit4] 2> 431135 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 431138 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[29 (1436162392205557760)]} 0 0
[junit4:junit4] 2> 431144 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1436162392208703488)]} 0 0
[junit4:junit4] 2> 431145 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[29 (1436162392208703488)]} 0 4
[junit4:junit4] 2> 431148 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[30 (1436162392216043520)]} 0 0
[junit4:junit4] 2> 431158 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1436162392222334976)]} 0 1
[junit4:junit4] 2> 431158 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1436162392222334976)]} 0 4
[junit4:junit4] 2> 431159 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[30]} 0 8
[junit4:junit4] 2> 431162 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[31 (1436162392230723584)]} 0 0
[junit4:junit4] 2> 431170 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1436162392235966464)]} 0 0
[junit4:junit4] 2> 431171 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1436162392235966464)]} 0 4
[junit4:junit4] 2> 431172 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4] 2> 431175 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[32 (1436162392244355072)]} 0 0
[junit4:junit4] 2> 431184 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1436162392249597952)]} 0 1
[junit4:junit4] 2> 431184 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1436162392249597952)]} 0 4
[junit4:junit4] 2> 431185 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[32]} 0 7
[junit4:junit4] 2> 431188 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[33 (1436162392257986560)]} 0 0
[junit4:junit4] 2> 431196 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1436162392263229440)]} 0 0
[junit4:junit4] 2> 431197 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1436162392263229440)]} 0 4
[junit4:junit4] 2> 431198 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[33]} 0 7
[junit4:junit4] 2> 431201 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[34 (1436162392271618048)]} 0 0
[junit4:junit4] 2> 431209 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1436162392276860928)]} 0 0
[junit4:junit4] 2> 431210 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1436162392276860928)]} 0 4
[junit4:junit4] 2> 431211 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[34]} 0 7
[junit4:junit4] 2> 431214 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[35 (1436162392285249536)]} 0 0
[junit4:junit4] 2> 431222 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1436162392290492416)]} 0 0
[junit4:junit4] 2> 431223 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1436162392290492416)]} 0 4
[junit4:junit4] 2> 431224 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[35]} 0 7
[junit4:junit4] 2> 431227 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[36 (1436162392298881024)]} 0 0
[junit4:junit4] 2> 431233 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1436162392302026752)]} 0 0
[junit4:junit4] 2> 431234 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[36 (1436162392302026752)]} 0 4
[junit4:junit4] 2> 431237 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[37 (1436162392309366784)]} 0 0
[junit4:junit4] 2> 431245 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1436162392314609664)]} 0 0
[junit4:junit4] 2> 431246 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1436162392314609664)]} 0 4
[junit4:junit4] 2> 431247 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4] 2> 431250 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[38 (1436162392322998272)]} 0 0
[junit4:junit4] 2> 431258 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1436162392328241152)]} 0 0
[junit4:junit4] 2> 431259 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1436162392328241152)]} 0 4
[junit4:junit4] 2> 431260 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[38]} 0 7
[junit4:junit4] 2> 431264 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[39 (1436162392336629760)]} 0 1
[junit4:junit4] 2> 431269 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1436162392339775488)]} 0 0
[junit4:junit4] 2> 431270 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[39 (1436162392339775488)]} 0 4
[junit4:junit4] 2> 431273 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[40 (1436162392347115520)]} 0 0
[junit4:junit4] 2> 431279 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1436162392350261248)]} 0 0
[junit4:junit4] 2> 431280 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[40 (1436162392350261248)]} 0 4
[junit4:junit4] 2> 431283 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[41 (1436162392357601280)]} 0 0
[junit4:junit4] 2> 431290 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1436162392360747008)]} 0 1
[junit4:junit4] 2> 431290 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[41 (1436162392360747008)]} 0 4
[junit4:junit4] 2> 431293 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[42 (1436162392368087040)]} 0 0
[junit4:junit4] 2> 431299 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1436162392371232768)]} 0 0
[junit4:junit4] 2> 431300 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[42 (1436162392371232768)]} 0 4
[junit4:junit4] 2> 431303 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[43 (1436162392378572800)]} 0 0
[junit4:junit4] 2> 431311 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1436162392383815680)]} 0 0
[junit4:junit4] 2> 431312 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1436162392383815680)]} 0 4
[junit4:junit4] 2> 431313 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[43]} 0 7
[junit4:junit4] 2> 431316 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[44 (1436162392392204288)]} 0 0
[junit4:junit4] 2> 431325 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1436162392397447168)]} 0 0
[junit4:junit4] 2> 431325 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1436162392397447168)]} 0 4
[junit4:junit4] 2> 431326 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[44]} 0 7
[junit4:junit4] 2> 431329 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[45 (1436162392405835776)]} 0 0
[junit4:junit4] 2> 431336 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1436162392408981504)]} 0 1
[junit4:junit4] 2> 431336 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[45 (1436162392408981504)]} 0 4
[junit4:junit4] 2> 431339 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[46 (1436162392416321536)]} 0 0
[junit4:junit4] 2> 431348 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1436162392421564416)]} 0 1
[junit4:junit4] 2> 431348 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1436162392421564416)]} 0 4
[junit4:junit4] 2> 431349 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[46]} 0 7
[junit4:junit4] 2> 431353 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[47 (1436162392431001600)]} 0 0
[junit4:junit4] 2> 431362 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1436162392436244480)]} 0 1
[junit4:junit4] 2> 431362 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[47 (1436162392436244480)]} 0 4
[junit4:junit4] 2> 431363 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[47]} 0 7
[junit4:junit4] 2> 431366 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[48 (1436162392444633088)]} 0 0
[junit4:junit4] 2> 431374 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1436162392449875968)]} 0 0
[junit4:junit4] 2> 431375 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1436162392449875968)]} 0 4
[junit4:junit4] 2> 431376 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[48]} 0 7
[junit4:junit4] 2> 431379 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[49 (1436162392458264576)]} 0 0
[junit4:junit4] 2> 431387 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1436162392464556032)]} 0 0
[junit4:junit4] 2> 431388 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[49 (1436162392464556032)]} 0 4
[junit4:junit4] 2> 431389 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[49]} 0 7
[junit4:junit4] 2> 431392 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[50 (1436162392471896064)]} 0 0
[junit4:junit4] 2> 431401 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1436162392478187520)]} 0 1
[junit4:junit4] 2> 431401 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1436162392478187520)]} 0 4
[junit4:junit4] 2> 431402 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[50]} 0 7
[junit4:junit4] 2> 431405 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[51 (1436162392485527552)]} 0 0
[junit4:junit4] 2> 431411 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1436162392488673280)]} 0 0
[junit4:junit4] 2> 431412 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[51 (1436162392488673280)]} 0 4
[junit4:junit4] 2> 431415 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[52 (1436162392496013312)]} 0 0
[junit4:junit4] 2> 431423 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1436162392501256192)]} 0 0
[junit4:junit4] 2> 431424 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[52 (1436162392501256192)]} 0 4
[junit4:junit4] 2> 431425 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[52]} 0 7
[junit4:junit4] 2> 431429 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[53 (1436162392509644800)]} 0 1
[junit4:junit4] 2> 431435 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1436162392512790528)]} 0 1
[junit4:junit4] 2> 431435 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[53 (1436162392512790528)]} 0 4
[junit4:junit4] 2> 431438 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[54 (1436162392520130560)]} 0 0
[junit4:junit4] 2> 431444 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1436162392523276288)]} 0 0
[junit4:junit4] 2> 431445 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[54 (1436162392523276288)]} 0 4
[junit4:junit4] 2> 431448 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[55 (1436162392530616320)]} 0 0
[junit4:junit4] 2> 431455 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1436162392533762048)]} 0 1
[junit4:junit4] 2> 431455 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[55 (1436162392533762048)]} 0 4
[junit4:junit4] 2> 431458 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[56 (1436162392541102080)]} 0 0
[junit4:junit4] 2> 431467 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1436162392546344960)]} 0 1
[junit4:junit4] 2> 431467 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1436162392546344960)]} 0 4
[junit4:junit4] 2> 431468 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[56]} 0 7
[junit4:junit4] 2> 431471 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[57 (1436162392554733568)]} 0 0
[junit4:junit4] 2> 431479 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1436162392559976448)]} 0 0
[junit4:junit4] 2> 431480 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1436162392559976448)]} 0 4
[junit4:junit4] 2> 431481 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[57]} 0 7
[junit4:junit4] 2> 431485 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[58 (1436162392568365056)]} 0 1
[junit4:junit4] 2> 431494 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1436162392574656512)]} 0 1
[junit4:junit4] 2> 431495 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1436162392574656512)]} 0 4
[junit4:junit4] 2> 431495 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[58]} 0 7
[junit4:junit4] 2> 431499 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[59 (1436162392583045120)]} 0 1
[junit4:junit4] 2> 431508 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1436162392589336576)]} 0 1
[junit4:junit4] 2> 431509 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1436162392589336576)]} 0 5
[junit4:junit4] 2> 431509 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[59]} 0 7
[junit4:junit4] 2> 431513 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[60 (1436162392597725184)]} 0 1
[junit4:junit4] 2> 431521 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1436162392602968064)]} 0 1
[junit4:junit4] 2> 431522 T1102 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1436162392602968064)]} 0 5
[junit4:junit4] 2> 431522 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[60]} 0 7
[junit4:junit4] 2> 431526 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[61 (1436162392611356672)]} 0 1
[junit4:junit4] 2> 431533 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1436162392616599552)]} 0 0
[junit4:junit4] 2> 431534 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1436162392616599552)]} 0 4
[junit4:junit4] 2> 431535 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[61]} 0 7
[junit4:junit4] 2> 431538 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[62 (1436162392624988160)]} 0 0
[junit4:junit4] 2> 431545 T1133 C153 P55312 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:57459/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1436162392628133888)]} 0 1
[junit4:junit4] 2> 431545 T1104 C155 P57459 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[62 (1436162392628133888)]} 0 4
[junit4:junit4] 2> 431548 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[63 (1436162392635473920)]} 0 0
[junit4:junit4] 2> 431554 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1436162392638619648)]} 0 0
[junit4:junit4] 2> 431555 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[63 (1436162392638619648)]} 0 4
[junit4:junit4] 2> 431558 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[64 (1436162392645959680)]} 0 0
[junit4:junit4] 2> 431564 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1436162392649105408)]} 0 0
[junit4:junit4] 2> 431565 T1086 C154 P61917 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[64 (1436162392649105408)]} 0 4
[junit4:junit4] 2> 431568 T1062 C156 P61913 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={wt=javabin&version=2} {add=[65 (1436162392656445440)]} 0 0
[junit4:junit4] 2> 431576 T1114 C152 P50621 oasup.LogUpdateProcessor.finish [collection1] webapp=/p/gi path=/update params={distrib.from=http://127.0.0.1:61917/p/gi/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1436162392661688320)]} 0 0
[junit4:junit4] 2> 431577 T1088 C154 P61917 oasup.LogUpdateProcessor.finish [collection1]
[...truncated too long message...]
tTest.splitShard(ShardSplitTest.java:208)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:679)
[junit4:junit4] 2> 687449 T1050 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 712932 T1141 oasu.PeerSync.handleResponse WARN PeerSync: core=collection1 url=http://127.0.0.1:55312/p/gi couldn't connect to http://127.0.0.1:57459/p/gi/collection1/, counting as success
[junit4:junit4] 2> 712933 T1141 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55312/p/gi DONE. sync succeeded
[junit4:junit4] 2> 712933 T1141 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 712935 T1141 oasc.ShardLeaderElectionContext.runLeaderProcess Sync was not a success but no one else is active! I am the leader
[junit4:junit4] 2> 712936 T1141 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55312/p/gi/collection1/
[junit4:junit4] 2> 712936 T1141 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@11b2a12e
[junit4:junit4] 2> 712937 T1141 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=866,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 712938 T1141 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 712938 T1141 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 712938 T1141 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 712941 T1141 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 712942 T1141 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 712942 T1141 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index;done=false>>]
[junit4:junit4] 2> 712942 T1141 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4/index
[junit4:junit4] 2> 712942 T1141 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4;done=false>>]
[junit4:junit4] 2> 712943 T1141 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369631160784/jetty4
[junit4:junit4] 2> 712943 T1141 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 712943 T1141 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89760147753533452-127.0.0.1:55312_p%2Fgi_collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 712944 T1141 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 712944 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 712944 T1141 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 712944 T1141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 712945 T1141 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 712945 T1141 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ca_ES, timezone=Asia/Pyongyang
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=3,free=129978744,total=471990272
[junit4:junit4] 2> NOTE: All tests run in this JVM: [EchoParamsTest, TestLMJelinekMercerSimilarityFactory, TestLuceneMatchVersion, TestUtils, UpdateParamsTest, DisMaxRequestHandlerTest, TestDFRSimilarityFactory, TestCoreDiscovery, IndexReaderFactoryTest, TestSuggestSpellingConverter, SpellCheckCollatorTest, TestMultiCoreConfBootstrap, NotRequiredUniqueKeyTest, TestJmxMonitoredMap, ZkNodePropsTest, TestShardHandlerFactory, TestReload, TestPostingsSolrHighlighter, SolrIndexConfigTest, MultiTermTest, TestOmitPositions, SimplePostToolTest, TestMergePolicyConfig, TestHashPartitioner, PingRequestHandlerTest, StandardRequestHandlerTest, TestSolrXMLSerializer, XmlUpdateRequestHandlerTest, ConvertedLegacyTest, TestStressLucene, TestCollationField, TestFieldTypeResource, TestZkChroot, XsltUpdateRequestHandlerTest, TestCSVResponseWriter, TestJmxIntegration, CollectionsAPIDistributedZkTest, TestSchemaNameResource, TermVectorComponentDistributedTest, TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, BasicDistributedZk2Test, TestFoldingMultitermQuery, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 309.22s, 1 test, 1 error <<< FAILURES!
[...truncated 655 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:887: There were test failures: 296 suites, 1234 tests, 1 error, 13 ignored (7 assumptions)
Total time: 50 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure