You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/05/25 23:19:48 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_21) - Build #
5830 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5830/
Java: 32bit/jdk1.7.0_21 -server -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Wrong doc count on shard1_0 expected:<339> but was:<338>
Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0 expected:<339> but was:<338>
at __randomizedtesting.SeedInfo.seed([183F4C3022B8A992:99D9C22855E7C9AE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:166)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:155)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9236 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 81375 T116 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 81382 T116 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1369515678614
[junit4:junit4] 2> 81386 T116 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 81389 T117 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 81588 T116 oasc.ZkTestServer.run start zk server on port:39749
[junit4:junit4] 2> 81654 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 81897 T123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c86afb name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 81898 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 81899 T116 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 81925 T118 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13edd7edc5d0000, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 81926 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 81948 T125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd3cb4 name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 81948 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 81950 T116 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 81954 T116 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 81957 T116 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 81960 T116 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 81963 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 81974 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 81981 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 81981 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 81986 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 81986 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 81990 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 81990 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 81993 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 81994 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 81997 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 81998 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 82001 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 82002 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 82005 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 82006 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 82009 T116 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 82010 T116 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 82291 T116 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 82398 T116 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42901
[junit4:junit4] 2> 82412 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 82413 T116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 82413 T116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246
[junit4:junit4] 2> 82414 T116 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/solr.xml
[junit4:junit4] 2> 82414 T116 oasc.CoreContainer.<init> New CoreContainer 13638379
[junit4:junit4] 2> 82415 T116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/'
[junit4:junit4] 2> 82415 T116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/'
[junit4:junit4] 2> 82641 T116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 82641 T116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 82642 T116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 82642 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 82643 T116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 82643 T116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 82643 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 82644 T116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 82644 T116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 82659 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 83283 T116 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 83284 T116 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39749/solr
[junit4:junit4] 2> 83290 T116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 83292 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 83294 T138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b061f2 name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 83294 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 83298 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 83306 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 83308 T140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19360f4 name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 83308 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 83313 T116 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 83321 T116 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 83329 T116 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 83332 T116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42901_
[junit4:junit4] 2> 83334 T116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42901_
[junit4:junit4] 2> 83339 T116 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 83350 T116 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 83354 T116 oasc.Overseer.start Overseer (id=89752579526688771-127.0.0.1:42901_-n_0000000000) starting
[junit4:junit4] 2> 83366 T116 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 83378 T142 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 83379 T116 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 83384 T116 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 83387 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 83400 T141 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 83423 T143 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/collection1
[junit4:junit4] 2> 83423 T143 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 83424 T143 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 83425 T143 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 83427 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/collection1/'
[junit4:junit4] 2> 83428 T143 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/collection1/lib/README' to classloader
[junit4:junit4] 2> 83428 T143 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 83474 T143 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 83532 T143 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 83534 T143 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 83556 T143 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 84025 T143 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 84028 T143 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 84030 T143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 84034 T143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 84069 T143 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 84070 T143 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369515679246/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data/
[junit4:junit4] 2> 84072 T143 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@129297f
[junit4:junit4] 2> 84073 T143 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 84073 T143 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data
[junit4:junit4] 2> 84074 T143 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data/index/
[junit4:junit4] 2> 84074 T143 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 84075 T143 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data/index
[junit4:junit4] 2> 84077 T143 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ee4262 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e6977),segFN=segments_1,generation=1}
[junit4:junit4] 2> 84077 T143 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 84091 T143 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 84092 T143 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 84093 T143 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 84094 T143 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 84095 T143 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 84095 T143 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 84096 T143 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 84097 T143 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 84097 T143 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 84101 T143 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 84105 T143 oass.SolrIndexSearcher.<init> Opening Searcher@da42b4 main
[junit4:junit4] 2> 84108 T143 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/control/data/tlog
[junit4:junit4] 2> 84116 T143 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 84116 T143 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 84121 T144 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@da42b4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 84124 T143 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 84124 T143 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 84906 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 84908 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42901_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42901"}
[junit4:junit4] 2> 84908 T141 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 84916 T141 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 84926 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 85127 T143 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 85127 T143 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42901 collection:control_collection shard:shard1
[junit4:junit4] 2> 85132 T143 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 85138 T143 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 85146 T143 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 85150 T143 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 85150 T143 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 85150 T143 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42901/collection1/
[junit4:junit4] 2> 85151 T143 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 85151 T143 oasc.SyncStrategy.syncToMe http://127.0.0.1:42901/collection1/ has no replicas
[junit4:junit4] 2> 85151 T143 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42901/collection1/
[junit4:junit4] 2> 85152 T143 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 86432 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 86442 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 86462 T143 oasc.ZkController.register We are http://127.0.0.1:42901/collection1/ and leader is http://127.0.0.1:42901/collection1/
[junit4:junit4] 2> 86464 T143 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42901
[junit4:junit4] 2> 86464 T143 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 86465 T143 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 86465 T143 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 86468 T143 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 86471 T116 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 86472 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 86495 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 86504 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 86508 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86510 T147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1159a6b name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86510 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86512 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 86516 T116 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 86605 T116 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 86607 T116 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56463
[junit4:junit4] 2> 86608 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 86608 T116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 86608 T116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748
[junit4:junit4] 2> 86609 T116 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/solr.xml
[junit4:junit4] 2> 86609 T116 oasc.CoreContainer.<init> New CoreContainer 2818658
[junit4:junit4] 2> 86610 T116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/'
[junit4:junit4] 2> 86610 T116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/'
[junit4:junit4] 2> 86669 T116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 86670 T116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 86670 T116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 86671 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 86671 T116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 86672 T116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 86672 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 86672 T116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 86673 T116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 86673 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 86682 T116 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 86683 T116 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39749/solr
[junit4:junit4] 2> 86683 T116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 86685 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86687 T158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@950332 name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86688 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86691 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 86699 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86702 T160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1be1629 name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86702 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86710 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 87720 T116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56463_
[junit4:junit4] 2> 87722 T116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56463_
[junit4:junit4] 2> 87724 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 87726 T160 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87727 T147 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87729 T140 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87735 T161 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/collection1
[junit4:junit4] 2> 87736 T161 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 87738 T161 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 87738 T161 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 87740 T161 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/collection1/'
[junit4:junit4] 2> 87741 T161 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/collection1/lib/README' to classloader
[junit4:junit4] 2> 87741 T161 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 87782 T161 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 87840 T161 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 87843 T161 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 87865 T161 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 87955 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 87959 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:42901__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42901_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42901"}
[junit4:junit4] 2> 87964 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 87964 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 87965 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 88316 T161 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 88319 T161 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 88320 T161 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 88325 T161 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 88378 T161 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 88378 T161 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369515683748/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1/
[junit4:junit4] 2> 88379 T161 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@129297f
[junit4:junit4] 2> 88380 T161 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 88380 T161 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1
[junit4:junit4] 2> 88381 T161 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1/index/
[junit4:junit4] 2> 88381 T161 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 88382 T161 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1/index
[junit4:junit4] 2> 88384 T161 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_1,generation=1}
[junit4:junit4] 2> 88385 T161 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 88389 T161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 88390 T161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 88391 T161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 88392 T161 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 88394 T161 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 88394 T161 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 88395 T161 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 88396 T161 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 88397 T161 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 88401 T161 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 88405 T161 oass.SolrIndexSearcher.<init> Opening Searcher@1b0bee3 main
[junit4:junit4] 2> 88406 T161 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty1/tlog
[junit4:junit4] 2> 88407 T161 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 88407 T161 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 88418 T162 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b0bee3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 88422 T161 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 88423 T161 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 89474 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 89475 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56463_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56463"}
[junit4:junit4] 2> 89475 T141 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 89476 T141 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 89482 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 89482 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 89482 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 90425 T161 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 90425 T161 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56463 collection:collection1 shard:shard1
[junit4:junit4] 2> 90426 T161 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 90435 T161 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 90438 T161 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 90438 T161 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 90438 T161 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56463/collection1/
[junit4:junit4] 2> 90439 T161 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 90439 T161 oasc.SyncStrategy.syncToMe http://127.0.0.1:56463/collection1/ has no replicas
[junit4:junit4] 2> 90439 T161 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56463/collection1/
[junit4:junit4] 2> 90439 T161 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 90990 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 91000 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 91000 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 91000 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 91048 T161 oasc.ZkController.register We are http://127.0.0.1:56463/collection1/ and leader is http://127.0.0.1:56463/collection1/
[junit4:junit4] 2> 91048 T161 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56463
[junit4:junit4] 2> 91048 T161 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 91048 T161 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 91049 T161 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 91052 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 91053 T116 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 91053 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 91054 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 91121 T116 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 91123 T116 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60571
[junit4:junit4] 2> 91123 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 91124 T116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 91124 T116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290
[junit4:junit4] 2> 91124 T116 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/solr.xml
[junit4:junit4] 2> 91125 T116 oasc.CoreContainer.<init> New CoreContainer 11760846
[junit4:junit4] 2> 91125 T116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/'
[junit4:junit4] 2> 91126 T116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/'
[junit4:junit4] 2> 91178 T116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 91179 T116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 91179 T116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 91180 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 91180 T116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 91180 T116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 91181 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 91181 T116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 91181 T116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 91182 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 91186 T116 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 91186 T116 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39749/solr
[junit4:junit4] 2> 91186 T116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 91187 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 91190 T174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c52eb name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 91190 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 91192 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 91196 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 91197 T176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f4b6fe name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 91197 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 91202 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 92206 T116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60571_
[junit4:junit4] 2> 92207 T116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60571_
[junit4:junit4] 2> 92209 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 92209 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 92210 T147 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 92211 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 92211 T176 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 92212 T140 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 92213 T160 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 92218 T177 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/collection1
[junit4:junit4] 2> 92219 T177 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 92219 T177 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 92220 T177 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 92221 T177 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/collection1/'
[junit4:junit4] 2> 92222 T177 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/collection1/lib/README' to classloader
[junit4:junit4] 2> 92222 T177 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 92252 T177 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 92287 T177 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 92289 T177 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 92304 T177 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 92508 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 92509 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:56463__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56463_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56463"}
[junit4:junit4] 2> 92514 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 92514 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 92514 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 92514 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 92658 T177 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 92660 T177 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 92662 T177 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 92666 T177 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 92700 T177 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 92701 T177 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369515688290/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2/
[junit4:junit4] 2> 92701 T177 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@129297f
[junit4:junit4] 2> 92702 T177 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 92702 T177 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2
[junit4:junit4] 2> 92703 T177 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2/index/
[junit4:junit4] 2> 92703 T177 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 92703 T177 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2/index
[junit4:junit4] 2> 92705 T177 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_1,generation=1}
[junit4:junit4] 2> 92705 T177 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 92708 T177 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 92708 T177 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 92709 T177 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 92710 T177 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 92710 T177 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 92711 T177 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 92711 T177 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 92712 T177 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 92712 T177 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 92715 T177 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 92717 T177 oass.SolrIndexSearcher.<init> Opening Searcher@9ba9ff main
[junit4:junit4] 2> 92718 T177 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty2/tlog
[junit4:junit4] 2> 92718 T177 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 92719 T177 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 92727 T178 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9ba9ff main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 92732 T177 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 92733 T177 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 94020 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 94021 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60571_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60571"}
[junit4:junit4] 2> 94022 T141 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 94022 T141 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 94027 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 94028 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 94028 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 94027 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 94735 T177 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 94735 T177 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60571 collection:collection1 shard:shard2
[junit4:junit4] 2> 94736 T177 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 94746 T177 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 94749 T177 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 94750 T177 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 94750 T177 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60571/collection1/
[junit4:junit4] 2> 94750 T177 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 94750 T177 oasc.SyncStrategy.syncToMe http://127.0.0.1:60571/collection1/ has no replicas
[junit4:junit4] 2> 94751 T177 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60571/collection1/
[junit4:junit4] 2> 94751 T177 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 95534 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 95543 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 95543 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 95544 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 95543 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 95560 T177 oasc.ZkController.register We are http://127.0.0.1:60571/collection1/ and leader is http://127.0.0.1:60571/collection1/
[junit4:junit4] 2> 95560 T177 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60571
[junit4:junit4] 2> 95561 T177 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 95561 T177 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 95561 T177 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 95564 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 95565 T116 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 95566 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 95566 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 95648 T116 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 95650 T116 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34417
[junit4:junit4] 2> 95651 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 95652 T116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 95652 T116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802
[junit4:junit4] 2> 95652 T116 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/solr.xml
[junit4:junit4] 2> 95653 T116 oasc.CoreContainer.<init> New CoreContainer 19485085
[junit4:junit4] 2> 95653 T116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/'
[junit4:junit4] 2> 95654 T116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/'
[junit4:junit4] 2> 95721 T116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 95722 T116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 95722 T116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 95723 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 95724 T116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 95724 T116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 95725 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 95725 T116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 95726 T116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 95726 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 95730 T116 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 95731 T116 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39749/solr
[junit4:junit4] 2> 95731 T116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 95732 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95734 T190 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11a4463 name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95734 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95736 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 95740 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95742 T192 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ca8a71 name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95742 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95746 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 96750 T116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34417_
[junit4:junit4] 2> 96752 T116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34417_
[junit4:junit4] 2> 96754 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 96754 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 96755 T176 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 96756 T147 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 96756 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 96756 T192 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 96756 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 96757 T140 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 96758 T160 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 96763 T193 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/collection1
[junit4:junit4] 2> 96763 T193 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 96764 T193 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 96764 T193 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 96766 T193 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/collection1/'
[junit4:junit4] 2> 96767 T193 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/collection1/lib/README' to classloader
[junit4:junit4] 2> 96767 T193 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 96809 T193 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 96858 T193 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 96859 T193 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 96878 T193 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 97052 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 97053 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60571__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60571_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60571"}
[junit4:junit4] 2> 97074 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 97074 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 97074 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 97074 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 97075 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 97229 T193 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 97232 T193 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 97234 T193 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 97243 T193 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 97286 T193 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 97286 T193 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369515692802/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/
[junit4:junit4] 2> 97286 T193 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@129297f
[junit4:junit4] 2> 97287 T193 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 97288 T193 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3
[junit4:junit4] 2> 97288 T193 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index/
[junit4:junit4] 2> 97288 T193 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 97289 T193 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index
[junit4:junit4] 2> 97290 T193 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@265d32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9d90b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 97290 T193 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 97293 T193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 97293 T193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 97294 T193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 97295 T193 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 97295 T193 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 97295 T193 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 97296 T193 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 97297 T193 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 97297 T193 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 97299 T193 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 97302 T193 oass.SolrIndexSearcher.<init> Opening Searcher@1b716f0 main
[junit4:junit4] 2> 97302 T193 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/tlog
[junit4:junit4] 2> 97303 T193 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 97303 T193 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 97312 T194 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b716f0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 97317 T193 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 97317 T193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 98581 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 98582 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34417_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34417"}
[junit4:junit4] 2> 98582 T141 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 98582 T141 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 98587 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 98587 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 98587 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 98588 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 98588 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 99319 T193 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 99319 T193 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34417 collection:collection1 shard:shard1
[junit4:junit4] 2> 99323 T193 oasc.ZkController.register We are http://127.0.0.1:34417/collection1/ and leader is http://127.0.0.1:56463/collection1/
[junit4:junit4] 2> 99323 T193 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34417
[junit4:junit4] 2> 99324 T193 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 99324 T193 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C201 name=collection1 org.apache.solr.core.SolrCore@95aea9 url=http://127.0.0.1:34417/collection1 node=127.0.0.1:34417_ C201_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34417_, base_url=http://127.0.0.1:34417}
[junit4:junit4] 2> 99328 T195 C201 P34417 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 99328 T195 C201 P34417 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 99328 T193 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 99329 T195 C201 P34417 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 99329 T195 C201 P34417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 99330 T116 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 99331 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 99330 T195 C201 P34417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 99331 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 99404 T116 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 99406 T116 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47846
[junit4:junit4] 2> 99407 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 99407 T116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 99408 T116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567
[junit4:junit4] 2> 99408 T116 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/solr.xml
[junit4:junit4] 2> 99408 T116 oasc.CoreContainer.<init> New CoreContainer 7146972
[junit4:junit4] 2> 99409 T116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/'
[junit4:junit4] 2> 99409 T116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/'
[junit4:junit4] 2> 99462 T116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 99463 T116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 99463 T116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 99464 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 99464 T116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 99464 T116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 99465 T116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 99465 T116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 99465 T116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 99466 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 99470 T116 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 99470 T116 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39749/solr
[junit4:junit4] 2> 99471 T116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 99472 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 99473 T207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@667cf5 name:ZooKeeperConnection Watcher:127.0.0.1:39749 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 99473 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 99475 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 99480 T116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 99482 T209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d51629 name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 99482 T116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 99487 T116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 99642 T155 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:34417__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 100093 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 100095 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34417__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34417_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34417"}
[junit4:junit4] 2> 100102 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100102 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100102 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100102 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100102 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100102 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100490 T116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47846_
[junit4:junit4] 2> 100492 T116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47846_
[junit4:junit4] 2> 100497 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100497 T147 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100497 T192 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100497 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 100499 T209 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100501 T140 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100500 T160 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100498 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 100498 T176 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 100498 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 100504 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 100502 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 100511 T210 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/collection1
[junit4:junit4] 2> 100511 T210 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 100512 T210 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 100513 T210 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 100515 T210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/collection1/'
[junit4:junit4] 2> 100516 T210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/collection1/lib/README' to classloader
[junit4:junit4] 2> 100517 T210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 100560 T210 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 100616 T210 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 100620 T210 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 100643 T210 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 100644 T155 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:34417__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 100645 T155 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:34417__collection1&state=recovering&nodeName=127.0.0.1:34417_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1005
[junit4:junit4] 2> 101089 T210 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 101092 T210 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 101094 T210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 101099 T210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 101137 T210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 101138 T210 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369515696567/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/
[junit4:junit4] 2> 101139 T210 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@129297f
[junit4:junit4] 2> 101140 T210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 101141 T210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4
[junit4:junit4] 2> 101142 T210 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index/
[junit4:junit4] 2> 101142 T210 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 101144 T210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index
[junit4:junit4] 2> 101147 T210 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea62c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@66945b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 101148 T210 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 101152 T210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 101153 T210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 101154 T210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 101155 T210 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 101157 T210 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 101157 T210 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 101158 T210 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 101159 T210 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 101160 T210 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 101164 T210 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 101168 T210 oass.SolrIndexSearcher.<init> Opening Searcher@1583c14 main
[junit4:junit4] 2> 101168 T210 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/tlog
[junit4:junit4] 2> 101169 T210 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 101169 T210 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 101177 T211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1583c14 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 101181 T210 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 101181 T210 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 101609 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101610 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47846_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47846"}
[junit4:junit4] 2> 101610 T141 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 101610 T141 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 101615 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 101616 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 101616 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 101616 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 101615 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 101615 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 102183 T210 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 102183 T210 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47846 collection:collection1 shard:shard2
[junit4:junit4] 2> 102187 T210 oasc.ZkController.register We are http://127.0.0.1:47846/collection1/ and leader is http://127.0.0.1:60571/collection1/
[junit4:junit4] 2> 102187 T210 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47846
[junit4:junit4] 2> 102187 T210 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 102187 T210 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C202 name=collection1 org.apache.solr.core.SolrCore@13f3251 url=http://127.0.0.1:47846/collection1 node=127.0.0.1:47846_ C202_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:47846_, base_url=http://127.0.0.1:47846}
[junit4:junit4] 2> 102188 T212 C202 P47846 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 102188 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 102189 T210 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 102189 T212 C202 P47846 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 102189 T212 C202 P47846 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 102190 T116 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 102190 T116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 102191 T212 C202 P47846 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 102191 T116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 102196 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 102198 T169 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:47846__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 102199 T116 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 102199 T116 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 102200 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C203 name=collection1 org.apache.solr.core.SolrCore@95aea9 url=http://127.0.0.1:34417/collection1 node=127.0.0.1:34417_ C203_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34417_, base_url=http://127.0.0.1:34417}
[junit4:junit4] 2> 102699 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56463/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 102702 T195 C203 P34417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 102707 T195 C203 P34417 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34417 START replicas=[http://127.0.0.1:56463/collection1/] nUpdates=100
[junit4:junit4] 2> 102710 T195 C203 P34417 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 102710 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 102711 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 102711 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 102711 T195 C203 P34417 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 102711 T195 C203 P34417 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56463/collection1/. core=collection1
[junit4:junit4] 2> 102712 T195 C203 P34417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C204 name=collection1 org.apache.solr.core.SolrCore@1c4f483 url=http://127.0.0.1:56463/collection1 node=127.0.0.1:56463_ C204_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56463_, base_url=http://127.0.0.1:56463, leader=true}
[junit4:junit4] 2> 102725 T154 C204 P56463 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 102728 T156 C204 P56463 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 102730 T156 C204 P56463 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_1,generation=1}
[junit4:junit4] 2> 102731 T156 C204 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 102731 T156 C204 P56463 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_2,generation=2}
[junit4:junit4] 2> 102732 T156 C204 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 102732 T156 C204 P56463 oass.SolrIndexSearcher.<init> Opening Searcher@1315b00 realtime
[junit4:junit4] 2> 102733 T156 C204 P56463 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 102733 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 102749 T195 C203 P34417 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 102749 T195 C203 P34417 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 102754 T195 C203 P34417 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 102779 T153 C204 P56463 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 102780 T153 C204 P56463 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=24
[junit4:junit4] 2> 102781 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 102781 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 102782 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 102785 T154 C204 P56463 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 102786 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 102787 T195 C203 P34417 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index.20130526000140018
[junit4:junit4] 2> 102787 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e8008 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d3983) fullCopy=false
[junit4:junit4] 2> 102792 T156 C204 P56463 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 102796 T195 C203 P34417 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 102801 T195 C203 P34417 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 102801 T195 C203 P34417 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 102803 T195 C203 P34417 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@265d32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9d90b),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@265d32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9d90b),segFN=segments_2,generation=2}
[junit4:junit4] 2> 102803 T195 C203 P34417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 102803 T195 C203 P34417 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 102804 T195 C203 P34417 oass.SolrIndexSearcher.<init> Opening Searcher@132ea08 main
[junit4:junit4] 2> 102805 T194 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@132ea08 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 102806 T195 C203 P34417 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index.20130526000140018 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index.20130526000140018;done=true>>]
[junit4:junit4] 2> 102806 T195 C203 P34417 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index.20130526000140018
[junit4:junit4] 2> 102807 T195 C203 P34417 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty3/index.20130526000140018
[junit4:junit4] 2> 102807 T195 C203 P34417 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 102807 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 102807 T195 C203 P34417 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 102807 T195 C203 P34417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 102809 T195 C203 P34417 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 103121 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 103123 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:47846__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47846_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47846"}
[junit4:junit4] 2> 103127 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34417__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34417_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34417"}
[junit4:junit4] 2> 103132 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103132 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103132 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103132 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103133 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103135 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 103199 T169 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:47846__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 103199 T169 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:47846__collection1&state=recovering&nodeName=127.0.0.1:47846_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 103202 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 104204 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C202_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:47846_, base_url=http://127.0.0.1:47846}
[junit4:junit4] 2> 105200 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60571/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 105201 T212 C202 P47846 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47846 START replicas=[http://127.0.0.1:60571/collection1/] nUpdates=100
[junit4:junit4] 2> 105201 T212 C202 P47846 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 105201 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 105202 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 105202 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 105202 T212 C202 P47846 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 105202 T212 C202 P47846 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60571/collection1/. core=collection1
[junit4:junit4] 2> 105203 T212 C202 P47846 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 105206 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C205 name=collection1 org.apache.solr.core.SolrCore@1b98c71 url=http://127.0.0.1:60571/collection1 node=127.0.0.1:60571_ C205_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60571_, base_url=http://127.0.0.1:60571, leader=true}
[junit4:junit4] 2> 105210 T169 C205 P60571 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 105216 T172 C205 P60571 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 105217 T172 C205 P60571 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_1,generation=1}
[junit4:junit4] 2> 105218 T172 C205 P60571 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 105219 T172 C205 P60571 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_2,generation=2}
[junit4:junit4] 2> 105219 T172 C205 P60571 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 105220 T172 C205 P60571 oass.SolrIndexSearcher.<init> Opening Searcher@17a7b8a realtime
[junit4:junit4] 2> 105220 T172 C205 P60571 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 105221 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 105222 T212 C202 P47846 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 105222 T212 C202 P47846 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 105225 T169 C205 P60571 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 105225 T169 C205 P60571 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 105226 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 105226 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 105226 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 105229 T172 C205 P60571 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 105230 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 105231 T212 C202 P47846 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index.20130526000142462
[junit4:junit4] 2> 105231 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@111c90b lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6d7f) fullCopy=false
[junit4:junit4] 2> 105234 T171 C205 P60571 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 105236 T212 C202 P47846 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 105237 T212 C202 P47846 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 105238 T212 C202 P47846 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 105239 T212 C202 P47846 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea62c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@66945b),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea62c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@66945b),segFN=segments_2,generation=2}
[junit4:junit4] 2> 105239 T212 C202 P47846 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 105240 T212 C202 P47846 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 105240 T212 C202 P47846 oass.SolrIndexSearcher.<init> Opening Searcher@118712b main
[junit4:junit4] 2> 105241 T211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@118712b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 105242 T212 C202 P47846 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index.20130526000142462 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index.20130526000142462;done=true>>]
[junit4:junit4] 2> 105242 T212 C202 P47846 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index.20130526000142462
[junit4:junit4] 2> 105242 T212 C202 P47846 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index.20130526000142462
[junit4:junit4] 2> 105243 T212 C202 P47846 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 105243 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 105243 T212 C202 P47846 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 105243 T212 C202 P47846 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 105245 T212 C202 P47846 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 106140 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 106141 T141 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:47846__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47846_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47846"}
[junit4:junit4] 2> 106146 T176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106147 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106147 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106147 T147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106147 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106146 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 106208 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 106209 T116 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C206 name=collection1 org.apache.solr.core.SolrCore@172ae37 url=http://127.0.0.1:42901/collection1 node=127.0.0.1:42901_ C206_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:42901_, base_url=http://127.0.0.1:42901, leader=true}
[junit4:junit4] 2> 106216 T134 C206 P42901 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 106218 T134 C206 P42901 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ee4262 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e6977),segFN=segments_1,generation=1}
[junit4:junit4] 2> 106218 T134 C206 P42901 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 106219 T134 C206 P42901 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ee4262 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e6977),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ee4262 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e6977),segFN=segments_2,generation=2}
[junit4:junit4] 2> 106220 T134 C206 P42901 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 106220 T134 C206 P42901 oass.SolrIndexSearcher.<init> Opening Searcher@1a2ab53 main
[junit4:junit4] 2> 106221 T134 C206 P42901 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 106222 T144 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a2ab53 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 106222 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 106232 T153 C204 P56463 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 106233 T153 C204 P56463 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d03ae1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1310af6),segFN=segments_3,generation=3}
[junit4:junit4] 2> 106234 T153 C204 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 106234 T153 C204 P56463 oass.SolrIndexSearcher.<init> Opening Searcher@1d4406c main
[junit4:junit4] 2> 106235 T153 C204 P56463 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 106236 T162 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d4406c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 106241 T153 C204 P56463 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34417/collection1/, StdNode: http://127.0.0.1:60571/collection1/, StdNode: http://127.0.0.1:47846/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 106247 T170 C205 P60571 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 106248 T170 C205 P60571 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b86231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ff3a57),segFN=segments_3,generation=3}
[junit4:junit4] 2> 106248 T170 C205 P60571 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 106249 T170 C205 P60571 oass.SolrIndexSearcher.<init> Opening Searcher@e13cd9 main
[junit4:junit4] 2> 106249 T170 C205 P60571 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 106251 T178 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e13cd9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 106251 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> C202_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47846_, base_url=http://127.0.0.1:47846}
[junit4:junit4] 2> 106252 T205 C202 P47846 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C203_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34417_, base_url=http://127.0.0.1:34417}
[junit4:junit4] 2> 106253 T188 C203 P34417 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 106253 T205 C202 P47846 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea62c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@66945b),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea62c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@66945b),segFN=segments_3,generation=3}
[junit4:junit4] 2> 106254 T188 C203 P34417 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@265d32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9d90b),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@265d32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9d90b),segFN=segments_3,generation=3}
[junit4:junit4] 2> 106255 T188 C203 P34417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 106254 T205 C202 P47846 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 106255 T188 C203 P34417 oass.SolrIndexSearcher.<init> Opening Searcher@1a7b4d0 main
[junit4:junit4] 2> 106256 T205 C202 P47846 oass.SolrIndexSearcher.<init> Opening Searcher@b7c1de main
[junit4:junit4] 2> 106256 T188 C203 P34417 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 106258 T211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b7c1de main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 106257 T194 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a7b4d0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 106257 T205 C202 P47846 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 106260 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 106260 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 106261 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 29
[junit4:junit4] 2> 106262 T116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 106268 T155 C204 P56463 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 106272 T187 C203 P34417 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 106274 T172 C205 P60571 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 106277 T203 C202 P47846 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 108320 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436041300450213888)} 0 23
[junit4:junit4] 2> 108328 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436041300468039680&update.from=http://127.0.0.1:56463/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436041300468039680)} 0 1
[junit4:junit4] 2> 108330 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436041300472233984&update.from=http://127.0.0.1:60571/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436041300472233984)} 0 0
[junit4:junit4] 2> 108331 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1436041300472233984)} 0 4
[junit4:junit4] 2> 108332 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436041300468039680)} 0 9
[junit4:junit4] 2> 108337 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1436041300481671168)]} 0 3
[junit4:junit4] 2> 108349 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1436041300491108352)]} 0 1
[junit4:junit4] 2> 108350 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1436041300491108352)]} 0 5
[junit4:junit4] 2> 108351 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 12
[junit4:junit4] 2> 108354 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1436041300500545536)]} 0 0
[junit4:junit4] 2> 108363 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1436041300506836992)]} 0 1
[junit4:junit4] 2> 108364 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1436041300506836992)]} 0 5
[junit4:junit4] 2> 108364 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 7
[junit4:junit4] 2> 108367 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1436041300514177024)]} 0 0
[junit4:junit4] 2> 108375 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1436041300519419904)]} 0 1
[junit4:junit4] 2> 108376 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1436041300519419904)]} 0 4
[junit4:junit4] 2> 108377 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 108380 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1436041300526759936)]} 0 1
[junit4:junit4] 2> 108385 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1436041300529905664)]} 0 0
[junit4:junit4] 2> 108386 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1436041300529905664)]} 0 4
[junit4:junit4] 2> 108389 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1436041300537245696)]} 0 0
[junit4:junit4] 2> 108395 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1436041300540391424)]} 0 1
[junit4:junit4] 2> 108395 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1436041300540391424)]} 0 4
[junit4:junit4] 2> 108398 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1436041300546682880)]} 0 0
[junit4:junit4] 2> 108404 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1436041300549828608)]} 0 1
[junit4:junit4] 2> 108405 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1436041300549828608)]} 0 4
[junit4:junit4] 2> 108408 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1436041300556120064)]} 0 1
[junit4:junit4] 2> 108416 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1436041300562411520)]} 0 0
[junit4:junit4] 2> 108417 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1436041300562411520)]} 0 4
[junit4:junit4] 2> 108418 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6]} 0 7
[junit4:junit4] 2> 108421 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1436041300570800128)]} 0 1
[junit4:junit4] 2> 108427 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1436041300573945856)]} 0 1
[junit4:junit4] 2> 108427 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1436041300573945856)]} 0 3
[junit4:junit4] 2> 108431 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1436041300580237312)]} 0 1
[junit4:junit4] 2> 108436 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1436041300584431616)]} 0 0
[junit4:junit4] 2> 108437 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1436041300584431616)]} 0 4
[junit4:junit4] 2> 108440 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1436041300590723072)]} 0 0
[junit4:junit4] 2> 108446 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1436041300593868800)]} 0 0
[junit4:junit4] 2> 108447 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1436041300593868800)]} 0 4
[junit4:junit4] 2> 108450 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1436041300601208832)]} 0 1
[junit4:junit4] 2> 108458 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1436041300606451712)]} 0 0
[junit4:junit4] 2> 108459 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1436041300606451712)]} 0 4
[junit4:junit4] 2> 108460 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10]} 0 8
[junit4:junit4] 2> 108463 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1436041300614840320)]} 0 0
[junit4:junit4] 2> 108472 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1436041300621131776)]} 0 0
[junit4:junit4] 2> 108472 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1436041300621131776)]} 0 4
[junit4:junit4] 2> 108473 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11]} 0 7
[junit4:junit4] 2> 108477 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1436041300628471808)]} 0 1
[junit4:junit4] 2> 108483 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1436041300632666112)]} 0 1
[junit4:junit4] 2> 108483 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1436041300632666112)]} 0 3
[junit4:junit4] 2> 108486 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1436041300638957568)]} 0 0
[junit4:junit4] 2> 108494 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1436041300644200448)]} 0 0
[junit4:junit4] 2> 108495 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1436041300644200448)]} 0 4
[junit4:junit4] 2> 108496 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13]} 0 7
[junit4:junit4] 2> 108499 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1436041300652589056)]} 0 0
[junit4:junit4] 2> 108505 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1436041300655734784)]} 0 1
[junit4:junit4] 2> 108506 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1436041300655734784)]} 0 5
[junit4:junit4] 2> 108509 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1436041300663074816)]} 0 0
[junit4:junit4] 2> 108517 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1436041300669366272)]} 0 0
[junit4:junit4] 2> 108518 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1436041300669366272)]} 0 3
[junit4:junit4] 2> 108519 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 108522 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1436041300676706304)]} 0 0
[junit4:junit4] 2> 108531 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1436041300681949184)]} 0 1
[junit4:junit4] 2> 108532 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1436041300681949184)]} 0 5
[junit4:junit4] 2> 108532 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16]} 0 7
[junit4:junit4] 2> 108536 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1436041300691386368)]} 0 0
[junit4:junit4] 2> 108545 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1436041300697677824)]} 0 0
[junit4:junit4] 2> 108546 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1436041300697677824)]} 0 4
[junit4:junit4] 2> 108547 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17]} 0 8
[junit4:junit4] 2> 108550 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1436041300706066432)]} 0 0
[junit4:junit4] 2> 108559 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1436041300712357888)]} 0 1
[junit4:junit4] 2> 108560 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1436041300712357888)]} 0 4
[junit4:junit4] 2> 108560 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18]} 0 7
[junit4:junit4] 2> 108563 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1436041300719697920)]} 0 0
[junit4:junit4] 2> 108571 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1436041300724940800)]} 0 1
[junit4:junit4] 2> 108572 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1436041300724940800)]} 0 4
[junit4:junit4] 2> 108572 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19]} 0 6
[junit4:junit4] 2> 108576 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436041300732280832)]} 0 1
[junit4:junit4] 2> 108582 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1436041300735426560)]} 0 1
[junit4:junit4] 2> 108583 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436041300735426560)]} 0 5
[junit4:junit4] 2> 108586 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1436041300743815168)]} 0 1
[junit4:junit4] 2> 108593 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1436041300748009472)]} 0 1
[junit4:junit4] 2> 108594 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1436041300748009472)]} 0 5
[junit4:junit4] 2> 108598 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1436041300756398080)]} 0 1
[junit4:junit4] 2> 108605 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1436041300760592384)]} 0 1
[junit4:junit4] 2> 108606 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1436041300760592384)]} 0 5
[junit4:junit4] 2> 108609 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1436041300767932416)]} 0 1
[junit4:junit4] 2> 108619 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1436041300773175296)]} 0 1
[junit4:junit4] 2> 108620 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1436041300773175296)]} 0 6
[junit4:junit4] 2> 108621 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23]} 0 9
[junit4:junit4] 2> 108624 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1436041300783661056)]} 0 0
[junit4:junit4] 2> 108633 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1436041300789952512)]} 0 1
[junit4:junit4] 2> 108634 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1436041300789952512)]} 0 4
[junit4:junit4] 2> 108635 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24]} 0 8
[junit4:junit4] 2> 108638 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1436041300798341120)]} 0 1
[junit4:junit4] 2> 108646 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1436041300803584000)]} 0 0
[junit4:junit4] 2> 108647 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1436041300803584000)]} 0 4
[junit4:junit4] 2> 108648 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4] 2> 108651 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1436041300811972608)]} 0 1
[junit4:junit4] 2> 108657 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1436041300815118336)]} 0 0
[junit4:junit4] 2> 108658 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1436041300815118336)]} 0 4
[junit4:junit4] 2> 108661 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1436041300822458368)]} 0 0
[junit4:junit4] 2> 108667 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1436041300825604096)]} 0 1
[junit4:junit4] 2> 108668 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1436041300825604096)]} 0 4
[junit4:junit4] 2> 108671 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1436041300832944128)]} 0 1
[junit4:junit4] 2> 108679 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1436041300838187008)]} 0 0
[junit4:junit4] 2> 108680 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1436041300838187008)]} 0 4
[junit4:junit4] 2> 108681 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 108685 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436041300846575616)]} 0 1
[junit4:junit4] 2> 108692 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1436041300850769920)]} 0 1
[junit4:junit4] 2> 108693 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436041300850769920)]} 0 5
[junit4:junit4] 2> 108697 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1436041300859158528)]} 0 1
[junit4:junit4] 2> 108703 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1436041300863352832)]} 0 1
[junit4:junit4] 2> 108704 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1436041300863352832)]} 0 5
[junit4:junit4] 2> 108708 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1436041300870692864)]} 0 1
[junit4:junit4] 2> 108717 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1436041300876984320)]} 0 1
[junit4:junit4] 2> 108717 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1436041300876984320)]} 0 4
[junit4:junit4] 2> 108718 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4] 2> 108721 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1436041300885372928)]} 0 0
[junit4:junit4] 2> 108727 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1436041300888518656)]} 0 0
[junit4:junit4] 2> 108728 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1436041300888518656)]} 0 4
[junit4:junit4] 2> 108732 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436041300895858688)]} 0 1
[junit4:junit4] 2> 108737 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1436041300900052992)]} 0 0
[junit4:junit4] 2> 108738 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436041300900052992)]} 0 4
[junit4:junit4] 2> 108741 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1436041300906344448)]} 0 0
[junit4:junit4] 2> 108749 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1436041300911587328)]} 0 0
[junit4:junit4] 2> 108750 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1436041300911587328)]} 0 4
[junit4:junit4] 2> 108751 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34]} 0 7
[junit4:junit4] 2> 108754 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1436041300919975936)]} 0 1
[junit4:junit4] 2> 108762 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1436041300925218816)]} 0 1
[junit4:junit4] 2> 108762 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1436041300925218816)]} 0 3
[junit4:junit4] 2> 108763 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35]} 0 6
[junit4:junit4] 2> 108766 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1436041300931510272)]} 0 1
[junit4:junit4] 2> 108773 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1436041300936753152)]} 0 1
[junit4:junit4] 2> 108774 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1436041300936753152)]} 0 4
[junit4:junit4] 2> 108774 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36]} 0 6
[junit4:junit4] 2> 108777 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1436041300944093184)]} 0 0
[junit4:junit4] 2> 108785 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1436041300949336064)]} 0 0
[junit4:junit4] 2> 108786 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1436041300949336064)]} 0 4
[junit4:junit4] 2> 108787 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4] 2> 108790 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1436041300957724672)]} 0 0
[junit4:junit4] 2> 108796 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1436041300960870400)]} 0 0
[junit4:junit4] 2> 108797 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1436041300960870400)]} 0 4
[junit4:junit4] 2> 108800 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1436041300967161856)]} 0 1
[junit4:junit4] 2> 108807 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1436041300972404736)]} 0 1
[junit4:junit4] 2> 108808 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1436041300972404736)]} 0 4
[junit4:junit4] 2> 108809 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39]} 0 7
[junit4:junit4] 2> 108812 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1436041300980793344)]} 0 1
[junit4:junit4] 2> 108822 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1436041300987084800)]} 0 1
[junit4:junit4] 2> 108823 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1436041300987084800)]} 0 5
[junit4:junit4] 2> 108824 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40]} 0 9
[junit4:junit4] 2> 108827 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1436041300996521984)]} 0 0
[junit4:junit4] 2> 108833 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1436041301000716288)]} 0 0
[junit4:junit4] 2> 108834 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1436041301000716288)]} 0 4
[junit4:junit4] 2> 108837 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1436041301007007744)]} 0 0
[junit4:junit4] 2> 108843 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1436041301010153472)]} 0 0
[junit4:junit4] 2> 108845 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1436041301010153472)]} 0 5
[junit4:junit4] 2> 108850 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1436041301019590656)]} 0 1
[junit4:junit4] 2> 108858 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1436041301025882112)]} 0 0
[junit4:junit4] 2> 108859 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1436041301025882112)]} 0 4
[junit4:junit4] 2> 108860 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43]} 0 7
[junit4:junit4] 2> 108863 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1436041301034270720)]} 0 1
[junit4:junit4] 2> 108869 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1436041301037416448)]} 0 1
[junit4:junit4] 2> 108870 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1436041301037416448)]} 0 4
[junit4:junit4] 2> 108875 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1436041301046853632)]} 0 1
[junit4:junit4] 2> 108884 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1436041301053145088)]} 0 1
[junit4:junit4] 2> 108885 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[45 (1436041301053145088)]} 0 5
[junit4:junit4] 2> 108886 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45]} 0 8
[junit4:junit4] 2> 108889 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1436041301061533696)]} 0 0
[junit4:junit4] 2> 108898 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1436041301067825152)]} 0 1
[junit4:junit4] 2> 108899 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1436041301067825152)]} 0 4
[junit4:junit4] 2> 108900 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46]} 0 8
[junit4:junit4] 2> 108903 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1436041301076213760)]} 0 0
[junit4:junit4] 2> 108910 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1436041301080408064)]} 0 1
[junit4:junit4] 2> 108911 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1436041301080408064)]} 0 5
[junit4:junit4] 2> 108915 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1436041301087748096)]} 0 1
[junit4:junit4] 2> 108923 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1436041301094039552)]} 0 1
[junit4:junit4] 2> 108924 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1436041301094039552)]} 0 4
[junit4:junit4] 2> 108925 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48]} 0 8
[junit4:junit4] 2> 108929 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1436041301102428160)]} 0 1
[junit4:junit4] 2> 108935 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1436041301106622464)]} 0 1
[junit4:junit4] 2> 108936 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1436041301106622464)]} 0 4
[junit4:junit4] 2> 108939 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1436041301113962496)]} 0 1
[junit4:junit4] 2> 108947 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1436041301119205376)]} 0 1
[junit4:junit4] 2> 108948 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1436041301119205376)]} 0 4
[junit4:junit4] 2> 108948 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50]} 0 6
[junit4:junit4] 2> 108951 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1436041301126545408)]} 0 0
[junit4:junit4] 2> 108959 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1436041301131788288)]} 0 1
[junit4:junit4] 2> 108960 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1436041301131788288)]} 0 4
[junit4:junit4] 2> 108960 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51]} 0 6
[junit4:junit4] 2> 108964 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1436041301139128320)]} 0 1
[junit4:junit4] 2> 108969 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1436041301142274048)]} 0 1
[junit4:junit4] 2> 108969 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1436041301142274048)]} 0 3
[junit4:junit4] 2> 108972 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436041301148565504)]} 0 0
[junit4:junit4] 2> 108977 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1436041301151711232)]} 0 0
[junit4:junit4] 2> 108978 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436041301151711232)]} 0 3
[junit4:junit4] 2> 108981 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1436041301156954112)]} 0 1
[junit4:junit4] 2> 108986 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1436041301160099840)]} 0 1
[junit4:junit4] 2> 108987 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1436041301160099840)]} 0 4
[junit4:junit4] 2> 108990 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1436041301166391296)]} 0 1
[junit4:junit4] 2> 108995 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1436041301169537024)]} 0 1
[junit4:junit4] 2> 108995 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1436041301169537024)]} 0 3
[junit4:junit4] 2> 108998 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436041301175828480)]} 0 0
[junit4:junit4] 2> 109003 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1436041301178974208)]} 0 0
[junit4:junit4] 2> 109004 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436041301178974208)]} 0 3
[junit4:junit4] 2> 109007 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1436041301185265664)]} 0 1
[junit4:junit4] 2> 109014 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1436041301189459968)]} 0 1
[junit4:junit4] 2> 109014 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1436041301189459968)]} 0 3
[junit4:junit4] 2> 109015 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57]} 0 6
[junit4:junit4] 2> 109018 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1436041301195751424)]} 0 1
[junit4:junit4] 2> 109025 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1436041301202042880)]} 0 0
[junit4:junit4] 2> 109026 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1436041301202042880)]} 0 3
[junit4:junit4] 2> 109027 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58]} 0 7
[junit4:junit4] 2> 109030 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1436041301209382912)]} 0 1
[junit4:junit4] 2> 109037 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1436041301214625792)]} 0 0
[junit4:junit4] 2> 109038 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1436041301214625792)]} 0 3
[junit4:junit4] 2> 109039 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59]} 0 6
[junit4:junit4] 2> 109042 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1436041301220917248)]} 0 1
[junit4:junit4] 2> 109049 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1436041301226160128)]} 0 1
[junit4:junit4] 2> 109049 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1436041301226160128)]} 0 3
[junit4:junit4] 2> 109050 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60]} 0 6
[junit4:junit4] 2> 109053 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1436041301232451584)]} 0 1
[junit4:junit4] 2> 109060 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1436041301237694464)]} 0 1
[junit4:junit4] 2> 109061 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1436041301237694464)]} 0 4
[junit4:junit4] 2> 109061 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61]} 0 6
[junit4:junit4] 2> 109064 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1436041301245034496)]} 0 0
[junit4:junit4] 2> 109071 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1436041301250277376)]} 0 0
[junit4:junit4] 2> 109072 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[62 (1436041301250277376)]} 0 3
[junit4:junit4] 2> 109073 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62]} 0 6
[junit4:junit4] 2> 109076 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1436041301256568832)]} 0 1
[junit4:junit4] 2> 109080 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1436041301259714560)]} 0 0
[junit4:junit4] 2> 109081 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1436041301259714560)]} 0 3
[junit4:junit4] 2> 109084 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1436041301264957440)]} 0 1
[junit4:junit4] 2> 109091 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1436041301270200320)]} 0 1
[junit4:junit4] 2> 109091 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[64 (1436041301270200320)]} 0 3
[junit4:junit4] 2> 109092 T171 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64]} 0 6
[junit4:junit4] 2> 109095 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1436041301277540352)]} 0 1
[junit4:junit4] 2> 109100 T185 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1436041301279637504)]} 0 1
[junit4:junit4] 2> 109101 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1436041301279637504)]} 0 3
[junit4:junit4] 2> 109103 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1436041301285928960)]} 0 0
[junit4:junit4] 2> 109109 T187 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[66 (1436041301289074688)]} 0 1
[junit4:junit4] 2> 109109 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1436041301289074688)]} 0 3
[junit4:junit4] 2> 109113 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1436041301295366144)]} 0 1
[junit4:junit4] 2> 109119 T203 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[67 (1436041301299560448)]} 0 1
[junit4:junit4] 2> 109120 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1436041301299560448)]} 0 4
[junit4:junit4] 2> 109123 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1436041301305851904)]} 0 1
[junit4:junit4] 2> 109130 T204 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[68 (1436041301311094784)]} 0 1
[junit4:junit4] 2> 109131 T169 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[68 (1436041301311094784)]} 0 4
[junit4:junit4] 2> 109131 T156 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68]} 0 6
[junit4:junit4] 2> 109134 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69 (1436041301318434816)]} 0 0
[junit4:junit4] 2> 109141 T205 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[69 (1436041301323677696)]} 0 0
[junit4:junit4] 2> 109142 T170 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[69 (1436041301323677696)]} 0 3
[junit4:junit4] 2> 109143 T155 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69]} 0 6
[junit4:junit4] 2> 109146 T136 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1436041301329969152)]} 0 1
[junit4:junit4] 2> 109151 T188 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[70 (1436041301333114880)]} 0 1
[junit4:junit4] 2> 109152 T153 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1436041301333114880)]} 0 4
[junit4:junit4] 2> 109155 T135 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1436041301340454912)]} 0 1
[junit4:junit4] 2> 109161 T202 C202 P47846 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60571/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[71 (1436041301343600640)]} 0 1
[junit4:junit4] 2> 109162 T172 C205 P60571 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1436041301343600640)]} 0 4
[junit4:junit4] 2> 109165 T133 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1436041301350940672)]} 0 0
[junit4:junit4] 2> 109171 T186 C203 P34417 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:56463/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[72 (1436041301354086400)]} 0 1
[junit4:junit4] 2> 109171 T154 C204 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1436041301354086400)]} 0 3
[junit4:junit4] 2> 109175 T134 C206 P42901 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[73 (1436041301360377856)]} 0 1
[junit4:junit4] 2> 109180 T185 C203 P34417 oasup.LogUpdateProcessor.finish [c
[...truncated too long message...]
letesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 250786 T116 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 250786 T116 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 250787 T116 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 250788 T116 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 250788 T116 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 250789 T116 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4;done=false>>]
[junit4:junit4] 2> 250789 T116 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4
[junit4:junit4] 2> 250789 T116 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index;done=false>>]
[junit4:junit4] 2> 250790 T116 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614/jetty4/index
[junit4:junit4] 2> 250790 T281 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89752579526688780-127.0.0.1:47846_-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 250791 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 250791 T209 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 250792 T209 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 250792 T209 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 250793 T276 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4:junit4] 2> 250813 T116 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 250869 T116 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 250871 T116 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39749 39749
[junit4:junit4] 2> 250973 T276 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1750bae name:ZooKeeperConnection Watcher:127.0.0.1:39749/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4] 2> 250974 T276 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
[junit4:junit4] 2> 250978 T116 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 42901
[junit4:junit4] 2> !!!! WARNING: best effort to remove /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369515678614 FAILED !!!!!
[junit4:junit4] 2> 250978 T116 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 250979 T116 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39749 39749
[junit4:junit4] 1> "core":"collection1_shard1_0_replica1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34417_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34417"}
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1_1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1_shard1_1_replica1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34417_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34417"}
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:60571_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60571"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89752579526688771-127.0.0.1:42901__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:42901_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:42901"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (6)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688776-127.0.0.1:60571_-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688784-127.0.0.1:39749_solr-n_0000000007 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688782-127.0.0.1:56463_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688780-127.0.0.1:47846_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688771-127.0.0.1:42901_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89752579526688778-127.0.0.1:34417_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89752579526688771-127.0.0.1:42901_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> 251007 T192 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 251008 T192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 251008 T192 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=183F4C3022B8A992 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_HN -Dtests.timezone=Europe/Minsk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 170s J0 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Wrong doc count on shard1_0 expected:<339> but was:<338>
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([183F4C3022B8A992:99D9C22855E7C9AE]:0)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:166)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:155)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 251132 T116 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 169761 T115 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=es_HN, timezone=Europe/Minsk
[junit4:junit4] 2> NOTE: Linux 3.2.0-41-generic i386/Oracle Corporation 1.7.0_21 (32-bit)/cpus=8,threads=1,free=89497272,total=213909504
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestRandomDVFaceting, QueryResultKeyTest, BinaryUpdateRequestHandlerTest, TestQueryUtils, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 170.57s, 1 test, 1 failure <<< FAILURES!
[...truncated 858 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:380: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:360: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1240: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:884: There were test failures: 295 suites, 1231 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 39 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
5831 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5831/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
2 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:59841/n_uu/onenodecollectioncore returned non ok status:404, message:Can not find: /n_uu/onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:59841/n_uu/onenodecollectioncore returned non ok status:404, message:Can not find: /n_uu/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([E3057B1F8FCF161F:62E3F507F8907623]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
REGRESSION: org.apache.solr.update.HardAutoCommitTest.testCommitWithin
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([E3057B1F8FCF161F:59D714670CE1F80A]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:525)
at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">3168</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><int name="intDefault">42</int><date name="timestamp">2013-05-25T23:13:45.843Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
</response>
request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:518)
... 41 more
Build Log:
[...truncated 9598 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 610685 T2393 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /n_uu/
[junit4:junit4] 2> 610689 T2393 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991
[junit4:junit4] 2> 610690 T2393 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 610690 T2394 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 610790 T2393 oasc.ZkTestServer.run start zk server on port:42389
[junit4:junit4] 2> 610792 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 610794 T2400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eb71a5e5 name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 610794 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 610794 T2393 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 610797 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 610798 T2402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d0bc0146 name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 610798 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 610798 T2393 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 610801 T2393 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 610803 T2393 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 610804 T2393 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 610806 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 610807 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 610810 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 610811 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 610813 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 610814 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 610816 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 610817 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 610818 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 610819 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 610821 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 610822 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 610823 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 610824 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 610826 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 610827 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 610829 T2393 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 610830 T2393 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 610897 T2393 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 610899 T2393 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52302
[junit4:junit4] 2> 610899 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 610900 T2393 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 610901 T2393 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134
[junit4:junit4] 2> 610901 T2393 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/solr.xml
[junit4:junit4] 2> 610902 T2393 oasc.CoreContainer.<init> New CoreContainer 58847713
[junit4:junit4] 2> 610903 T2393 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/'
[junit4:junit4] 2> 610903 T2393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/'
[junit4:junit4] 2> 610940 T2393 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 610941 T2393 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 610941 T2393 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 610942 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 610943 T2393 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 610943 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 610944 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 610944 T2393 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 610945 T2393 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 610946 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 610951 T2393 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 610952 T2393 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42389/solr
[junit4:junit4] 2> 610952 T2393 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 610953 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 610955 T2413 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e66f6fb2 name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 610955 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 610956 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 610962 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 610963 T2415 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50887436 name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 610963 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 610964 T2393 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 610967 T2393 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 610969 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 610970 T2393 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52302_n_uu
[junit4:junit4] 2> 610972 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52302_n_uu
[junit4:junit4] 2> 610974 T2393 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 610979 T2393 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 610982 T2393 oasc.Overseer.start Overseer (id=89753071320694787-127.0.0.1:52302_n_uu-n_0000000000) starting
[junit4:junit4] 2> 610985 T2393 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 610991 T2417 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 610992 T2393 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 610995 T2393 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 610997 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 610999 T2416 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 611001 T2418 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/collection1
[junit4:junit4] 2> 611002 T2418 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 611002 T2418 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 611002 T2418 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 611003 T2418 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/collection1/'
[junit4:junit4] 2> 611004 T2418 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/collection1/lib/README' to classloader
[junit4:junit4] 2> 611004 T2418 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 611027 T2418 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 611068 T2418 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 611070 T2418 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 611074 T2418 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 611387 T2418 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 611392 T2418 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 611395 T2418 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 611404 T2418 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 611407 T2418 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 611411 T2418 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 611412 T2418 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 611412 T2418 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 611412 T2418 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 611413 T2418 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 611414 T2418 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 611414 T2418 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 611414 T2418 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1369523183134/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/
[junit4:junit4] 2> 611415 T2418 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 611415 T2418 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 611416 T2418 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data
[junit4:junit4] 2> 611416 T2418 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index/
[junit4:junit4] 2> 611417 T2418 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 611417 T2418 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index
[junit4:junit4] 2> 611419 T2418 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a64d3279 lockFactory=org.apache.lucene.store.NativeFSLockFactory@dc430063),segFN=segments_1,generation=1}
[junit4:junit4] 2> 611419 T2418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 611420 T2418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 611421 T2418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 611421 T2418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 611422 T2418 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 611422 T2418 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 611423 T2418 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 611423 T2418 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 611423 T2418 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 611424 T2418 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 611425 T2418 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 611427 T2418 oass.SolrIndexSearcher.<init> Opening Searcher@4dcb3db4 main
[junit4:junit4] 2> 611427 T2418 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/tlog
[junit4:junit4] 2> 611428 T2418 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 611428 T2418 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 611432 T2419 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4dcb3db4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 611432 T2418 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 611433 T2418 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 612501 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 612502 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52302/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52302_n_uu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 612502 T2416 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 612503 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 612505 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 613434 T2418 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 613435 T2418 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52302/n_uu collection:control_collection shard:shard1
[junit4:junit4] 2> 613436 T2418 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 613446 T2418 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 613450 T2418 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 613451 T2418 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 613452 T2418 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52302/n_uu/collection1/
[junit4:junit4] 2> 613454 T2418 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 613456 T2418 oasc.SyncStrategy.syncToMe http://127.0.0.1:52302/n_uu/collection1/ has no replicas
[junit4:junit4] 2> 613456 T2418 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52302/n_uu/collection1/
[junit4:junit4] 2> 613457 T2418 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 614009 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 614019 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 614063 T2418 oasc.ZkController.register We are http://127.0.0.1:52302/n_uu/collection1/ and leader is http://127.0.0.1:52302/n_uu/collection1/
[junit4:junit4] 2> 614064 T2418 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52302/n_uu
[junit4:junit4] 2> 614064 T2418 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 614064 T2418 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 614065 T2418 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 614066 T2418 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 614067 T2393 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 614068 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 614068 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 614074 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 614076 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 614077 T2422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d7672cba name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 614077 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 614079 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 614080 T2393 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 614155 T2393 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 614157 T2393 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55419
[junit4:junit4] 2> 614158 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 614159 T2393 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 614159 T2393 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383
[junit4:junit4] 2> 614160 T2393 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/solr.xml
[junit4:junit4] 2> 614161 T2393 oasc.CoreContainer.<init> New CoreContainer 2013579373
[junit4:junit4] 2> 614162 T2393 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/'
[junit4:junit4] 2> 614163 T2393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/'
[junit4:junit4] 2> 614209 T2393 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 614210 T2393 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 614210 T2393 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 614211 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 614211 T2393 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 614212 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 614213 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 614213 T2393 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 614214 T2393 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 614215 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 614219 T2393 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 614220 T2393 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42389/solr
[junit4:junit4] 2> 614221 T2393 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 614222 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 614223 T2433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a4d5fd7b name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 614223 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 614225 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 614230 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 614231 T2435 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c9dbcc90 name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 614231 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 614233 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 615236 T2393 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55419_n_uu
[junit4:junit4] 2> 615237 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55419_n_uu
[junit4:junit4] 2> 615240 T2422 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 615240 T2415 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 615240 T2435 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 615241 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 615256 T2436 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/collection1
[junit4:junit4] 2> 615256 T2436 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 615257 T2436 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 615257 T2436 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 615258 T2436 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/collection1/'
[junit4:junit4] 2> 615259 T2436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/collection1/lib/README' to classloader
[junit4:junit4] 2> 615259 T2436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 615283 T2436 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 615313 T2436 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 615314 T2436 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 615318 T2436 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 615523 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 615524 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52302/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52302_n_uu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 615526 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 615526 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 615526 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 615619 T2436 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 615623 T2436 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 615626 T2436 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 615634 T2436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 615637 T2436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 615641 T2436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 615641 T2436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 615642 T2436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 615642 T2436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 615643 T2436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 615643 T2436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 615644 T2436 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 615644 T2436 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/
[junit4:junit4] 2> 615644 T2436 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 615645 T2436 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 615646 T2436 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1
[junit4:junit4] 2> 615646 T2436 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index/
[junit4:junit4] 2> 615646 T2436 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 615647 T2436 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index
[junit4:junit4] 2> 615648 T2436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76c97554 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a058c8fb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 615648 T2436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 615650 T2436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 615650 T2436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 615651 T2436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 615651 T2436 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 615652 T2436 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 615652 T2436 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 615653 T2436 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 615653 T2436 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 615654 T2436 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 615655 T2436 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 615656 T2436 oass.SolrIndexSearcher.<init> Opening Searcher@edfca7c8 main
[junit4:junit4] 2> 615657 T2436 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/tlog
[junit4:junit4] 2> 615658 T2436 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 615658 T2436 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 615662 T2437 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@edfca7c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 615664 T2436 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 615664 T2436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 617030 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 617031 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 617031 T2416 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 617032 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 617037 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 617037 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 617038 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 617666 T2436 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 617666 T2436 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55419/n_uu collection:collection1 shard:shard1
[junit4:junit4] 2> 617667 T2436 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 617676 T2436 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 617678 T2436 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 617679 T2436 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 617679 T2436 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55419/n_uu/collection1/
[junit4:junit4] 2> 617679 T2436 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 617680 T2436 oasc.SyncStrategy.syncToMe http://127.0.0.1:55419/n_uu/collection1/ has no replicas
[junit4:junit4] 2> 617680 T2436 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55419/n_uu/collection1/
[junit4:junit4] 2> 617680 T2436 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 618542 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 618547 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 618547 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 618547 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 618586 T2436 oasc.ZkController.register We are http://127.0.0.1:55419/n_uu/collection1/ and leader is http://127.0.0.1:55419/n_uu/collection1/
[junit4:junit4] 2> 618587 T2436 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55419/n_uu
[junit4:junit4] 2> 618587 T2436 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 618588 T2436 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 618588 T2436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 618589 T2436 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 618591 T2393 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 618591 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 618592 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 618661 T2393 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 618662 T2393 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59841
[junit4:junit4] 2> 618663 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 618664 T2393 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 618664 T2393 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899
[junit4:junit4] 2> 618665 T2393 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/solr.xml
[junit4:junit4] 2> 618666 T2393 oasc.CoreContainer.<init> New CoreContainer -681137677
[junit4:junit4] 2> 618666 T2393 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/'
[junit4:junit4] 2> 618667 T2393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/'
[junit4:junit4] 2> 618719 T2393 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 618719 T2393 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 618720 T2393 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 618720 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 618721 T2393 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 618722 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 618722 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 618723 T2393 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 618723 T2393 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 618724 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 618729 T2393 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 618730 T2393 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42389/solr
[junit4:junit4] 2> 618731 T2393 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 618732 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 618733 T2449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b1f62228 name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 618734 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 618735 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 618740 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 618742 T2451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c3145bc6 name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 618742 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 618745 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 619748 T2393 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59841_n_uu
[junit4:junit4] 2> 619750 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59841_n_uu
[junit4:junit4] 2> 619752 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 619752 T2451 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 619752 T2415 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 619752 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 619753 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 619754 T2422 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 619755 T2435 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 619757 T2452 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/collection1
[junit4:junit4] 2> 619757 T2452 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 619758 T2452 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 619758 T2452 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 619760 T2452 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/collection1/'
[junit4:junit4] 2> 619760 T2452 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/collection1/lib/README' to classloader
[junit4:junit4] 2> 619761 T2452 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 619787 T2452 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 619820 T2452 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 619822 T2452 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 619828 T2452 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 620050 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 620051 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 620053 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 620053 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 620053 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 620053 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 620151 T2452 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 620156 T2452 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 620158 T2452 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 620166 T2452 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 620170 T2452 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 620173 T2452 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 620174 T2452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 620174 T2452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 620175 T2452 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 620176 T2452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 620176 T2452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 620176 T2452 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 620177 T2452 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1369523190899/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/
[junit4:junit4] 2> 620177 T2452 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 620186 T2452 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 620187 T2452 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2
[junit4:junit4] 2> 620188 T2452 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index/
[junit4:junit4] 2> 620188 T2452 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 620188 T2452 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index
[junit4:junit4] 2> 620190 T2452 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b07666d8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8565c54f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 620190 T2452 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 620192 T2452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 620192 T2452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 620192 T2452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 620193 T2452 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 620194 T2452 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 620194 T2452 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 620194 T2452 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 620195 T2452 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 620195 T2452 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 620196 T2452 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 620198 T2452 oass.SolrIndexSearcher.<init> Opening Searcher@54d36989 main
[junit4:junit4] 2> 620199 T2452 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/tlog
[junit4:junit4] 2> 620199 T2452 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 620200 T2452 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 620204 T2453 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54d36989 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 620205 T2452 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 620206 T2452 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 621558 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 621558 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59841/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:59841_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 621559 T2416 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 621559 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 621563 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 621563 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 621563 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 621563 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 622207 T2452 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 622208 T2452 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59841/n_uu collection:collection1 shard:shard2
[junit4:junit4] 2> 622209 T2452 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 622213 T2452 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 622214 T2452 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 622215 T2452 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 622215 T2452 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59841/n_uu/collection1/
[junit4:junit4] 2> 622216 T2452 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 622216 T2452 oasc.SyncStrategy.syncToMe http://127.0.0.1:59841/n_uu/collection1/ has no replicas
[junit4:junit4] 2> 622217 T2452 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59841/n_uu/collection1/
[junit4:junit4] 2> 622217 T2452 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 623066 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 623072 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 623072 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 623072 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 623072 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 623122 T2452 oasc.ZkController.register We are http://127.0.0.1:59841/n_uu/collection1/ and leader is http://127.0.0.1:59841/n_uu/collection1/
[junit4:junit4] 2> 623123 T2452 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59841/n_uu
[junit4:junit4] 2> 623123 T2452 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 623124 T2452 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 623124 T2452 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 623126 T2452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 623127 T2393 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 623128 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 623128 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 623201 T2393 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 623203 T2393 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60699
[junit4:junit4] 2> 623204 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 623205 T2393 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 623205 T2393 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435
[junit4:junit4] 2> 623206 T2393 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/solr.xml
[junit4:junit4] 2> 623206 T2393 oasc.CoreContainer.<init> New CoreContainer -489540076
[junit4:junit4] 2> 623207 T2393 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/'
[junit4:junit4] 2> 623208 T2393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/'
[junit4:junit4] 2> 623248 T2393 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 623249 T2393 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 623250 T2393 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 623250 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 623251 T2393 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 623252 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 623252 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 623253 T2393 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 623254 T2393 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 623254 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 623260 T2393 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 623261 T2393 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42389/solr
[junit4:junit4] 2> 623262 T2393 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 623263 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 623264 T2465 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b0a3a7ac name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 623265 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 623266 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 623271 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 623273 T2467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9dd32f9 name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 623273 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 623276 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 624278 T2393 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60699_n_uu
[junit4:junit4] 2> 624280 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60699_n_uu
[junit4:junit4] 2> 624282 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 624282 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 624282 T2415 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 624283 T2467 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 624282 T2451 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 624283 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624284 T2435 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 624284 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624288 T2468 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/collection1
[junit4:junit4] 2> 624288 T2468 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 624288 T2422 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 624288 T2468 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 624289 T2468 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 624291 T2468 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/collection1/'
[junit4:junit4] 2> 624292 T2468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/collection1/lib/README' to classloader
[junit4:junit4] 2> 624292 T2468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 624319 T2468 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 624352 T2468 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 624354 T2468 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 624358 T2468 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 624576 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 624576 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59841/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:59841_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 624579 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624579 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624579 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624579 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624579 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 624716 T2468 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 624722 T2468 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 624725 T2468 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 624733 T2468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 624737 T2468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 624740 T2468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 624741 T2468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 624741 T2468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 624742 T2468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 624743 T2468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 624743 T2468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 624743 T2468 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 624744 T2468 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1369523195435/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/
[junit4:junit4] 2> 624744 T2468 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 624744 T2468 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 624745 T2468 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3
[junit4:junit4] 2> 624745 T2468 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index/
[junit4:junit4] 2> 624746 T2468 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 624746 T2468 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index
[junit4:junit4] 2> 624747 T2468 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a288f44 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fec09cb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 624748 T2468 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 624749 T2468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 624750 T2468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 624750 T2468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 624751 T2468 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 624751 T2468 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 624752 T2468 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 624752 T2468 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 624753 T2468 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 624753 T2468 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 624754 T2468 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 624756 T2468 oass.SolrIndexSearcher.<init> Opening Searcher@6a20f023 main
[junit4:junit4] 2> 624756 T2468 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/tlog
[junit4:junit4] 2> 624757 T2468 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 624757 T2468 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 624761 T2469 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a20f023 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 624762 T2468 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 624763 T2468 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 626083 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 626084 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60699/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:60699_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 626084 T2416 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 626085 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 626088 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 626088 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 626088 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 626088 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 626088 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 626809 T2468 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 626809 T2468 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60699/n_uu collection:collection1 shard:shard1
[junit4:junit4] 2> 626811 T2468 oasc.ZkController.register We are http://127.0.0.1:60699/n_uu/collection1/ and leader is http://127.0.0.1:55419/n_uu/collection1/
[junit4:junit4] 2> 626812 T2468 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60699/n_uu
[junit4:junit4] 2> 626812 T2468 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 626812 T2468 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1296 name=collection1 org.apache.solr.core.SolrCore@ae0eb5cd url=http://127.0.0.1:60699/n_uu/collection1 node=127.0.0.1:60699_n_uu C1296_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60699/n_uu, state=down, node_name=127.0.0.1:60699_n_uu, collection=collection1, core=collection1}
[junit4:junit4] 2> 626813 T2470 C1296 P60699 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 626813 T2468 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 626813 T2470 C1296 P60699 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 626814 T2470 C1296 P60699 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 626814 T2470 C1296 P60699 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 626814 T2393 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 626815 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 626815 T2470 C1296 P60699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 626816 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 626822 T2428 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 626888 T2393 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 626890 T2393 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40285
[junit4:junit4] 2> 626891 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 626892 T2393 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 626892 T2393 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123
[junit4:junit4] 2> 626893 T2393 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/solr.xml
[junit4:junit4] 2> 626893 T2393 oasc.CoreContainer.<init> New CoreContainer 1570074178
[junit4:junit4] 2> 626894 T2393 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/'
[junit4:junit4] 2> 626895 T2393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/'
[junit4:junit4] 2> 626933 T2393 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 626934 T2393 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 626934 T2393 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 626935 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 626936 T2393 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 626936 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 626937 T2393 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 626937 T2393 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 626938 T2393 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 626939 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 626944 T2393 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 626945 T2393 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42389/solr
[junit4:junit4] 2> 626945 T2393 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 626946 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 626948 T2482 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@867b1d0a name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 626948 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 626949 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 626955 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 626956 T2484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f5f1a5c name:ZooKeeperConnection Watcher:127.0.0.1:42389/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 626956 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 626959 T2393 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 627591 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 627591 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60699/n_uu",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:60699_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 627594 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627594 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627594 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627594 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627594 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627594 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627823 T2428 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 627823 T2428 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=4&wt=javabin&nodeName=127.0.0.1:60699_n_uu&core=collection1} status=0 QTime=1001
[junit4:junit4] 2> 627961 T2393 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40285_n_uu
[junit4:junit4] 2> 627963 T2393 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40285_n_uu
[junit4:junit4] 2> 627964 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627964 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627965 T2484 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627964 T2451 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627966 T2415 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627966 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 627964 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 627967 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 627966 T2435 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627966 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 627967 T2422 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627969 T2467 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 627970 T2485 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/collection1
[junit4:junit4] 2> 627970 T2485 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 627971 T2485 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 627971 T2485 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 627972 T2485 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/collection1/'
[junit4:junit4] 2> 627973 T2485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/collection1/lib/README' to classloader
[junit4:junit4] 2> 627973 T2485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 627997 T2485 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 628026 T2485 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 628027 T2485 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 628031 T2485 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 628365 T2485 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 628370 T2485 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 628372 T2485 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 628381 T2485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 628384 T2485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 628388 T2485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 628388 T2485 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 628389 T2485 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 628389 T2485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 628390 T2485 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 628390 T2485 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 628391 T2485 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 628391 T2485 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1369523199123/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/
[junit4:junit4] 2> 628391 T2485 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 628392 T2485 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 628393 T2485 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4
[junit4:junit4] 2> 628393 T2485 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index/
[junit4:junit4] 2> 628393 T2485 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 628394 T2485 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index
[junit4:junit4] 2> 628395 T2485 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fbfefbe6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ee96a8c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 628396 T2485 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 628397 T2485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 628397 T2485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 628398 T2485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 628399 T2485 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 628399 T2485 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 628399 T2485 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 628400 T2485 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 628400 T2485 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 628401 T2485 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 628402 T2485 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 628404 T2485 oass.SolrIndexSearcher.<init> Opening Searcher@5b2fd5f4 main
[junit4:junit4] 2> 628404 T2485 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/tlog
[junit4:junit4] 2> 628405 T2485 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 628405 T2485 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 628409 T2486 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b2fd5f4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 628411 T2485 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 628411 T2485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 629097 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 629098 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40285/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:40285_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 629098 T2416 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 629098 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 629102 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629102 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629102 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629102 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629102 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629104 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 629412 T2485 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 629413 T2485 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40285/n_uu collection:collection1 shard:shard2
[junit4:junit4] 2> 629416 T2485 oasc.ZkController.register We are http://127.0.0.1:40285/n_uu/collection1/ and leader is http://127.0.0.1:59841/n_uu/collection1/
[junit4:junit4] 2> 629416 T2485 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40285/n_uu
[junit4:junit4] 2> 629417 T2485 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 629417 T2485 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1297 name=collection1 org.apache.solr.core.SolrCore@154a351 url=http://127.0.0.1:40285/n_uu/collection1 node=127.0.0.1:40285_n_uu C1297_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:40285/n_uu, state=down, node_name=127.0.0.1:40285_n_uu, collection=collection1, core=collection1}
[junit4:junit4] 2> 629418 T2487 C1297 P40285 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 629418 T2485 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 629418 T2487 C1297 P40285 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 629419 T2487 C1297 P40285 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 629420 T2487 C1297 P40285 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 629420 T2393 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 629421 T2393 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 629420 T2487 C1297 P40285 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 629421 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 629427 T2446 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 629428 T2393 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 629429 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 629435 T2429 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/onenodecollectioncore
[junit4:junit4] 2> 629435 T2429 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 629436 T2429 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 629437 T2429 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 629438 T2429 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 629438 T2429 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 629440 T2429 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 629442 T2429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/onenodecollectioncore/'
[junit4:junit4] 2> 629467 T2429 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 629498 T2429 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 629500 T2429 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 629504 T2429 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1298 name=collection1 org.apache.solr.core.SolrCore@ae0eb5cd url=http://127.0.0.1:60699/n_uu/collection1 node=127.0.0.1:60699_n_uu C1298_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60699/n_uu, state=recovering, node_name=127.0.0.1:60699_n_uu, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 629824 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55419/n_uu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 629825 T2470 C1298 P60699 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60699/n_uu START replicas=[http://127.0.0.1:55419/n_uu/collection1/] nUpdates=100
[junit4:junit4] 2> 629825 T2470 C1298 P60699 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 629825 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 629826 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 629826 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 629826 T2470 C1298 P60699 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 629826 T2470 C1298 P60699 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55419/n_uu/collection1/. core=collection1
[junit4:junit4] 2> 629827 T2470 C1298 P60699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1299 name=collection1 org.apache.solr.core.SolrCore@381cddd3 url=http://127.0.0.1:55419/n_uu/collection1 node=127.0.0.1:55419_n_uu C1299_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55419/n_uu, state=active, node_name=127.0.0.1:55419_n_uu, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 629830 T2430 C1299 P55419 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 629833 T2428 C1299 P55419 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 629834 T2428 C1299 P55419 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76c97554 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a058c8fb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 629834 T2428 C1299 P55419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 629835 T2428 C1299 P55419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76c97554 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a058c8fb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76c97554 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a058c8fb),segFN=segments_2,generation=2}
[junit4:junit4] 2> 629836 T2428 C1299 P55419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 629837 T2428 C1299 P55419 oass.SolrIndexSearcher.<init> Opening Searcher@7b9eb241 realtime
[junit4:junit4] 2> 629837 T2428 C1299 P55419 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 629838 T2428 C1299 P55419 oasup.LogUpdateProcessor.finish [collection1] webapp=/n_uu path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 6
[junit4:junit4] 2> 629839 T2470 C1298 P60699 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 629839 T2429 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 629839 T2470 C1298 P60699 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 629841 T2428 C1299 P55419 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 629841 T2428 C1299 P55419 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 629842 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 629842 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 629842 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 629844 T2430 C1299 P55419 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 629844 T2429 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 629844 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 629846 T2470 C1298 P60699 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index.20130526120642147
[junit4:junit4] 2> 629846 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@759f721c lockFactory=org.apache.lucene.store.NativeFSLockFactory@e13af816) fullCopy=false
[junit4:junit4] 2> 629847 T2429 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 629848 T2431 C1299 P55419 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 629849 T2470 C1298 P60699 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 629849 T2470 C1298 P60699 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 629850 T2470 C1298 P60699 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 629851 T2470 C1298 P60699 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a288f44 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fec09cb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a288f44 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fec09cb3),segFN=segments_2,generation=2}
[junit4:junit4] 2> 629851 T2470 C1298 P60699 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 629851 T2470 C1298 P60699 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 629852 T2470 C1298 P60699 oass.SolrIndexSearcher.<init> Opening Searcher@25d31043 main
[junit4:junit4] 2> 629853 T2469 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25d31043 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 629853 T2470 C1298 P60699 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index.20130526120642147 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index.20130526120642147;done=true>>]
[junit4:junit4] 2> 629854 T2470 C1298 P60699 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index.20130526120642147
[junit4:junit4] 2> 629854 T2470 C1298 P60699 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index.20130526120642147
[junit4:junit4] 2> 629854 T2470 C1298 P60699 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 629855 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 629855 T2470 C1298 P60699 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 629855 T2470 C1298 P60699 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 629856 T2470 C1298 P60699 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 629856 T2429 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 629862 T2429 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 629867 T2429 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 629868 T2429 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 629868 T2429 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 629869 T2429 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 629870 T2429 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 629871 T2429 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 629872 T2429 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 629872 T2429 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1369523186383/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/
[junit4:junit4] 2> 629873 T2429 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fa441e07
[junit4:junit4] 2> 629874 T2429 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 629875 T2429 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection
[junit4:junit4] 2> 629876 T2429 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index/
[junit4:junit4] 2> 629877 T2429 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 629878 T2429 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index
[junit4:junit4] 2> 629879 T2429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4323d14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@24b30352),segFN=segments_1,generation=1}
[junit4:junit4] 2> 629880 T2429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 629882 T2429 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 629883 T2429 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 629884 T2429 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 629885 T2429 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 629886 T2429 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 629887 T2429 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 629887 T2429 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 629888 T2429 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 629889 T2429 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 629890 T2429 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 629892 T2429 oass.SolrIndexSearcher.<init> Opening Searcher@c8f6abe6 main
[junit4:junit4] 2> 629893 T2429 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/tlog
[junit4:junit4] 2> 629894 T2429 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 629895 T2429 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 629898 T2491 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@c8f6abe6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 629900 T2429 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 630606 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 630607 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40285/n_uu",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:40285_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 630609 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60699/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:60699_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 630611 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 630611 T2416 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 630612 T2416 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 630614 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630614 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630614 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630614 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630614 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630614 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 630901 T2429 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 630902 T2429 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:55419/n_uu collection:onenodecollection shard:shard1
[junit4:junit4] 2> 630903 T2429 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 630909 T2429 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 630911 T2429 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 630912 T2429 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 630913 T2429 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55419/n_uu/onenodecollectioncore/
[junit4:junit4] 2> 630914 T2429 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 630915 T2429 oasc.SyncStrategy.syncToMe http://127.0.0.1:55419/n_uu/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 630916 T2429 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55419/n_uu/onenodecollectioncore/
[junit4:junit4] 2> 630917 T2429 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 631428 T2446 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 631428 T2446 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=5&wt=javabin&nodeName=127.0.0.1:40285_n_uu&core=collection1} status=0 QTime=2001
[junit4:junit4] 2> 632120 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 632126 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632126 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632126 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632126 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632126 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632126 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 632174 T2429 oasc.ZkController.register We are http://127.0.0.1:55419/n_uu/onenodecollectioncore/ and leader is http://127.0.0.1:55419/n_uu/onenodecollectioncore/
[junit4:junit4] 2> 632174 T2429 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:55419/n_uu
[junit4:junit4] 2> 632175 T2429 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 632175 T2429 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 632177 T2429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 632178 T2429 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=2743
[junit4:junit4] 2> 632179 T2393 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 632180 T2393 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 633182 T2393 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1300 name=collection1 org.apache.solr.core.SolrCore@154a351 url=http://127.0.0.1:40285/n_uu/collection1 node=127.0.0.1:40285_n_uu C1300_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:40285/n_uu, state=recovering, node_name=127.0.0.1:40285_n_uu, collection=collection1, shard=shard2, core=collection1}
[junit4:junit4] 2> 633429 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59841/n_uu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 633430 T2487 C1300 P40285 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40285/n_uu START replicas=[http://127.0.0.1:59841/n_uu/collection1/] nUpdates=100
[junit4:junit4] 2> 633430 T2487 C1300 P40285 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 633430 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 633431 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 633431 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 633431 T2487 C1300 P40285 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 633432 T2487 C1300 P40285 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59841/n_uu/collection1/. core=collection1
[junit4:junit4] 2> 633432 T2487 C1300 P40285 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1301 name=collection1 org.apache.solr.core.SolrCore@5f42eb96 url=http://127.0.0.1:59841/n_uu/collection1 node=127.0.0.1:59841_n_uu C1301_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:59841/n_uu, state=active, node_name=127.0.0.1:59841_n_uu, collection=collection1, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 633436 T2446 C1301 P59841 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 633439 T2444 C1301 P59841 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 633441 T2444 C1301 P59841 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b07666d8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8565c54f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 633442 T2444 C1301 P59841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 633443 T2444 C1301 P59841 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b07666d8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8565c54f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b07666d8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8565c54f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 633444 T2444 C1301 P59841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 633446 T2444 C1301 P59841 oass.SolrIndexSearcher.<init> Opening Searcher@ad02acd1 realtime
[junit4:junit4] 2> 633447 T2444 C1301 P59841 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 633448 T2444 C1301 P59841 oasup.LogUpdateProcessor.finish [collection1] webapp=/n_uu path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 9
[junit4:junit4] 2> 633449 T2487 C1300 P40285 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 633449 T2487 C1300 P40285 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 633450 T2445 C1301 P59841 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 633451 T2445 C1301 P59841 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 633452 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 633452 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 633452 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 633454 T2446 C1301 P59841 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 633455 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 633455 T2487 C1300 P40285 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index.20130526120645757
[junit4:junit4] 2> 633456 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@12aabce2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7c84e5b) fullCopy=false
[junit4:junit4] 2> 633457 T2444 C1301 P59841 oasc.SolrCore.execute [collection1] webapp=/n_uu path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 633459 T2487 C1300 P40285 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 633460 T2487 C1300 P40285 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 633460 T2487 C1300 P40285 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 633461 T2487 C1300 P40285 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fbfefbe6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ee96a8c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fbfefbe6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ee96a8c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 633462 T2487 C1300 P40285 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 633462 T2487 C1300 P40285 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 633463 T2487 C1300 P40285 oass.SolrIndexSearcher.<init> Opening Searcher@1add827e main
[junit4:junit4] 2> 633464 T2486 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1add827e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 633464 T2487 C1300 P40285 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index.20130526120645757 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index.20130526120645757;done=true>>]
[junit4:junit4] 2> 633464 T2487 C1300 P40285 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index.20130526120645757
[junit4:junit4] 2> 633465 T2487 C1300 P40285 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index.20130526120645757
[junit4:junit4] 2> 633465 T2487 C1300 P40285 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 633465 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 633466 T2487 C1300 P40285 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 633466 T2487 C1300 P40285 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 633479 T2487 C1300 P40285 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 633631 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 633632 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55419_n_uu_onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 633636 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40285/n_uu",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:40285_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 633640 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 633640 T2422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 633640 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 633640 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 633640 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 633640 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 634184 T2393 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 634185 T2393 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 634186 T2393 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 634196 T2393 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 634197 T2495 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71e4449b name:ZooKeeperConnection Watcher:127.0.0.1:42389 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 634198 T2393 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 634232 T2393 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52302
[junit4:junit4] 2> 634232 T2393 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=58847713
[junit4:junit4] 2> 635145 T2416 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 635146 T2416 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52302/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52302_n_uu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 635149 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 635149 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 635149 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 635149 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 635149 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 636234 T2393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 636235 T2393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 636236 T2393 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@eb0bfb9d
[junit4:junit4] 2> 636239 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 636240 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 636240 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 636241 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 636241 T2393 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 636242 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 636243 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data;done=false>>]
[junit4:junit4] 2> 636243 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data
[junit4:junit4] 2> 636244 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index;done=false>>]
[junit4:junit4] 2> 636244 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/control/data/index
[junit4:junit4] 2> 636245 T2416 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89753071320694787-127.0.0.1:52302_n_uu-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 636246 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 636246 T2484 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636246 T2451 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636247 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636247 T2467 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636247 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636247 T2435 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 636250 T2435 oasc.Overseer.start Overseer (id=89753071320694790-127.0.0.1:55419_n_uu-n_0000000001) starting
[junit4:junit4] 2> 636255 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 636255 T2497 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 636256 T2496 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 636257 T2435 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636258 T2496 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 636259 T2496 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52302/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52302_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 636266 T2393 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/n_uu,null}
[junit4:junit4] 2> 636278 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636278 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636278 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636278 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636318 T2393 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 55419
[junit4:junit4] 2> 636319 T2393 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2013579373
[junit4:junit4] 2> 637546 T2415 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 637546 T2415 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 637547 T2415 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 637782 T2496 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 637783 T2496 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55419_n_uu_onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 637786 T2496 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 637789 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 637789 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 637789 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 637789 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639321 T2393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 639321 T2393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 639323 T2393 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 639324 T2393 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@381cddd3
[junit4:junit4] 2> 639327 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 639328 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 639328 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 639329 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 639330 T2393 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 639331 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 639331 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index;done=false>>]
[junit4:junit4] 2> 639332 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1/index
[junit4:junit4] 2> 639332 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1;done=false>>]
[junit4:junit4] 2> 639333 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty1
[junit4:junit4] 2> 639334 T2393 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@4afae388
[junit4:junit4] 2> 639338 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 639339 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 639339 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 639340 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 639341 T2393 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 639341 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 639342 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index;done=false>>]
[junit4:junit4] 2> 639342 T2393 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection/index
[junit4:junit4] 2> 639343 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection;done=false>>]
[junit4:junit4] 2> 639343 T2393 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1369523182991/onenodecollection
[junit4:junit4] 2> 639344 T2496 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89753071320694790-127.0.0.1:55419_n_uu-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 639345 T2435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639345 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639345 T2435 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 639345 T2484 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 639346 T2435 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 639346 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 639347 T2467 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 639346 T2451 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 639348 T2467 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 639349 T2451 oasc.Overseer.start Overseer (id=89753071320694792-127.0.0.1:59841_n_uu-n_0000000002) starting
[junit4:junit4] 2> 639350 T2467 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 639350 T2467 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 639350 T2467 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 639350 T2467 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60699/n_uu/collection1/
[junit4:junit4] 2> 639351 T2467 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 639351 T2467 oasc.SyncStrategy.syncToMe http://127.0.0.1:60699/n_uu/collection1/ has no replicas
[junit4:junit4] 2> 639351 T2467 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60699/n_uu/collection1/
[junit4:junit4] 2> 639352 T2467 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 639366 T2393 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/n_uu,null}
[junit4:junit4] 2> 639371 T2499 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 639372 T2451 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 639372 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 639373 T2498 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 639376 T2498 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 639377 T2498 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 639380 T2498 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55419/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55419_n_uu_onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:55419_n_uu",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 639391 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 639391 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 639391 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 639418 T2393 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 59841
[junit4:junit4] 2> 639419 T2393 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-681137677
[junit4:junit4] 2> 640420 T2393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 640421 T2393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 640423 T2393 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5f42eb96
[junit4:junit4] 2> 640425 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 640426 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 640427 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 640427 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 640428 T2393 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 640429 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 640429 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2;done=false>>]
[junit4:junit4] 2> 640430 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2
[junit4:junit4] 2> 640430 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index;done=false>>]
[junit4:junit4] 2> 640431 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty2/index
[junit4:junit4] 2> 640431 T2498 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89753071320694792-127.0.0.1:59841_n_uu-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 640433 T2484 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 640433 T2467 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 640435 T2484 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 640435 T2484 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 640435 T2484 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 640435 T2467 oasc.Overseer.start Overseer (id=89753071320694794-127.0.0.1:60699_n_uu-n_0000000003) starting
[junit4:junit4] 2> 640436 T2484 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40285/n_uu/collection1/
[junit4:junit4] 2> 640436 T2484 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40285/n_uu START replicas=[http://127.0.0.1:59841/n_uu/collection1/] nUpdates=100
[junit4:junit4] 2> 640437 T2484 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40285/n_uu DONE. We have no versions. sync failed.
[junit4:junit4] 2> 640437 T2484 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 640437 T2484 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 640438 T2484 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40285/n_uu/collection1/
[junit4:junit4] 2> 640438 T2484 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 640439 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 640439 T2502 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 640439 T2501 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 640440 T2467 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 640441 T2484 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 640441 T2501 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 640442 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 640442 T2501 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59841/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:59841_n_uu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 640443 T2393 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/n_uu,null}
[junit4:junit4] 2> 640445 T2501 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59841/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:59841_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 640453 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 640454 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 640495 T2393 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 60699
[junit4:junit4] 2> 640496 T2393 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-489540076
[junit4:junit4] 2> 641497 T2393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 641498 T2393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 641500 T2393 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@ae0eb5cd
[junit4:junit4] 2> 641503 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 641504 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 641504 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 641505 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 641506 T2393 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 641506 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 641507 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3;done=false>>]
[junit4:junit4] 2> 641507 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3
[junit4:junit4] 2> 641508 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index;done=false>>]
[junit4:junit4] 2> 641509 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty3/index
[junit4:junit4] 2> 641509 T2501 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89753071320694794-127.0.0.1:60699_n_uu-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 641510 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 641510 T2467 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 641510 T2484 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 641511 T2467 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 641511 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 641512 T2484 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 641514 T2484 oasc.Overseer.start Overseer (id=89753071320694796-127.0.0.1:40285_n_uu-n_0000000004) starting
[junit4:junit4] 2> 641517 T2504 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 641517 T2503 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 641519 T2503 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 641519 T2503 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60699/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:60699_n_uu",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 641521 T2503 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60699/n_uu",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:60699_n_uu",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 641523 T2484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 641530 T2393 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/n_uu,null}
[junit4:junit4] 2> 641583 T2393 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 40285
[junit4:junit4] 2> 641583 T2393 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1570074178
[junit4:junit4] 2> 641732 T2451 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 641733 T2451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 641733 T2451 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 642585 T2393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 642586 T2393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 642591 T2393 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@154a351
[junit4:junit4] 2> 642599 T2393 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 642600 T2393 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 642601 T2393 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 642601 T2393 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 642602 T2393 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 642603 T2393 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 642604 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4;done=false>>]
[junit4:junit4] 2> 642604 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4
[junit4:junit4] 2> 642605 T2393 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index;done=false>>]
[junit4:junit4] 2> 642606 T2393 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1369523182991/jetty4/index
[junit4:junit4] 2> 642607 T2503 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89753071320694796-127.0.0.1:40285_n_uu-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 642628 T2393 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/n_uu,null}
[junit4:junit4] 2> 642685 T2393 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 642686 T2393 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:42389 42389
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> / (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr (7)
[junit4:junit4] 1> /solr/configs (1)
[junit4:junit4] 1> /solr/configs/conf1 (9)
[junit4:junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89753071320694787-127.0.0.1:52302_n_uu-n_0000000000"}
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89753071320694796-127.0.0.1:40285_n_uu-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89753071320694792-127.0.0.1:59841_n_uu-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89753071320694790-127.0.0.1:55419_n_uu-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89753071320694787-127.0.0.1:52302_n_uu-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89753071320694794-127.0.0.1:60699_n_uu-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections (3)
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leade
[...truncated too long message...]
junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:780)
[junit4:junit4] 2>
[junit4:junit4] 2> 1056752 T3704 oas.SolrTestCaseJ4.tearDown ###Ending testCommitWithin
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=E3057B1F8FCF161F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pt_BR -Dtests.timezone=PNT -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 5.00s J0 | HardAutoCommitTest.testCommitWithin <<<
[junit4:junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([E3057B1F8FCF161F:59D714670CE1F80A]:0)
[junit4:junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:525)
[junit4:junit4] > at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:79)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:780)
[junit4:junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
[junit4:junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4:junit4] > <response>
[junit4:junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">3168</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><int name="range_facet_si">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="range_facet_sl"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><int name="intDefault">42</int><date name="timestamp">2013-05-25T23:13:45.843Z</date><arr name="multiDefault"><str>muLti-Default</str></arr></doc></result>
[junit4:junit4] > </response>
[junit4:junit4] > request was:q=id:529&start=0&qt=standard&version=2.2&rows=20
[junit4:junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:518)
[junit4:junit4] > ... 41 more
[junit4:junit4] 2> 1056762 T3704 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 1056762 T3704 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=35505764
[junit4:junit4] 2> 1056763 T3704 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@c0765d03
[junit4:junit4] 2> 1056774 T3704 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=1,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 1056777 T3704 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1056778 T3704 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1056779 T3704 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1056782 T3704 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1056783 T3704 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1056783 T3704 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374;done=false>>]
[junit4:junit4] 2> 1056784 T3704 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374
[junit4:junit4] 2> 1056785 T3704 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374/index;done=false>>]
[junit4:junit4] 2> 1056785 T3704 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-HardAutoCommitTest-1369523623374/index
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=pt_BR, timezone=PNT
[junit4:junit4] 2> NOTE: Linux 3.2.0-41-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=76942144,total=201195520
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestFieldCollectionResource, CircularListTest, TestDynamicFieldCollectionResource, PrimitiveFieldTypeTest, SOLR749Test, BadComponentTest, CoreAdminHandlerTest, ShardRoutingTest, TestMaxScoreQueryParser, ResourceLoaderTest, TestQueryUtils, TestCloudManagedSchema, TestUniqueKeyFieldResource, PreAnalyzedFieldTest, PluginInfoTest, ClusterStateUpdateTest, TestManagedSchemaFieldResource, FileBasedSpellCheckerTest, AnalysisAfterCoreReloadTest, IndexSchemaTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreTest, SuggesterFSTTest, PeerSyncTest, AlternateDirectoryTest, FastVectorHighlighterTest, TestJoin, RAMDirectoryFactoryTest, StatelessScriptUpdateProcessorFactoryTest, TestPseudoReturnFields, TestXIncludeConfig, TestCharFilters, TestFuzzyAnalyzedSuggestions, TestPluginEnable, TestSort, TestDocSet, DirectUpdateHandlerOptimizeTest, TestConfig, TestDynamicFieldResource, TestTrie, TestCopyFieldCollectionResource, ZkControllerTest, SuggesterTest, UUIDFieldTest, MBeansHandlerTest, TestSolrQueryParserDefaultOperatorResource, PolyFieldTest, CSVRequestHandlerTest, TestFaceting, SliceStateTest, FieldMutatingUpdateProcessorTest, TestRandomDVFaceting, TestClassNameShortening, TestSurroundQueryParser, ReturnFieldsTest, TestFastWriter, BadCopyFieldTest, DistributedSpellCheckComponentTest, TestSolrDeletionPolicy1, TestSolrCoreProperties, TestPropInjectDefaults, TestReversedWildcardFilterFactory, LukeRequestHandlerTest, TestRemoteStreaming, DocValuesTest, DateFieldTest, UpdateRequestProcessorFactoryTest, TestSweetSpotSimilarityFactory, TestReplicationHandler, CacheHeaderTest, SpellingQueryConverterTest, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, OverseerTest, BasicZkTest, TestRandomFaceting, TestRecovery, TestRealTimeGet, HardAutoCommitTest]
[junit4:junit4] Completed on J0 in 5.76s, 1 test, 1 error <<< FAILURES!
[...truncated 403 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:380: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:373: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1240: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:884: There were test failures: 295 suites, 1231 tests, 2 errors, 14 ignored (8 assumptions)
Total time: 45 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure