You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/24 10:52:32 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_25) - Build #
6285 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6285/
Java: 32bit/jdk1.7.0_25 -client -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:34390 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:34390 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([7785BBA5FA80CFCF:F66335BD8DDFAFF3]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 10064 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4:junit4] 2> 1080310 T4474 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1080314 T4474 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ShardSplitTest-1372063737327
[junit4:junit4] 2> 1080315 T4474 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1080316 T4475 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1080416 T4474 oasc.ZkTestServer.run start zk server on port:39152
[junit4:junit4] 2> 1080417 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1080435 T4481 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@811ea3 name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1080436 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1080436 T4474 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1080439 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1080440 T4483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f2dda4 name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1080441 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1080441 T4474 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1080445 T4474 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1080447 T4474 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1080449 T4474 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1080450 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1080451 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1080453 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1080454 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1080456 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1080457 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1080459 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1080459 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1080461 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1080461 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1080463 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1080464 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1080465 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1080466 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1080468 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1080468 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1080470 T4474 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1080471 T4474 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1080532 T4474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1080535 T4474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50857
[junit4:junit4] 2> 1080535 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1080536 T4474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1080536 T4474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485
[junit4:junit4] 2> 1080536 T4474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/solr.xml
[junit4:junit4] 2> 1080537 T4474 oasc.CoreContainer.<init> New CoreContainer 75305
[junit4:junit4] 2> 1080537 T4474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/'
[junit4:junit4] 2> 1080537 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/'
[junit4:junit4] 2> 1080594 T4474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1080595 T4474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1080595 T4474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1080595 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1080596 T4474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1080596 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1080596 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1080597 T4474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1080597 T4474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1080597 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1080600 T4474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1080601 T4474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39152/solr
[junit4:junit4] 2> 1080601 T4474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1080602 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1080604 T4494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e39293 name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1080604 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1080605 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1080608 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1080609 T4496 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25e198 name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1080610 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1080611 T4474 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1080614 T4474 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1080616 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1080617 T4474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50857_
[junit4:junit4] 2> 1080618 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50857_
[junit4:junit4] 2> 1080620 T4474 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1080623 T4474 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1080625 T4474 oasc.Overseer.start Overseer (id=89919569089593347-127.0.0.1:50857_-n_0000000000) starting
[junit4:junit4] 2> 1080627 T4474 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1080630 T4498 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1080630 T4474 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1080632 T4474 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1080633 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1080635 T4497 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1080638 T4499 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1080639 T4499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1082138 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1082139 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50857_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50857"}
[junit4:junit4] 2> 1082139 T4497 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1082139 T4497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1082143 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1082640 T4499 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/collection1
[junit4:junit4] 2> 1082641 T4499 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1082642 T4499 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1082642 T4499 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1082643 T4499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/collection1/'
[junit4:junit4] 2> 1082644 T4499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/collection1/lib/README' to classloader
[junit4:junit4] 2> 1082645 T4499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1082686 T4499 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1082728 T4499 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1082729 T4499 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1082734 T4499 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1083142 T4499 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1083145 T4499 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1083146 T4499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1083149 T4499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1083164 T4499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1083164 T4499 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1372063737485/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data/
[junit4:junit4] 2> 1083164 T4499 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@104cf88
[junit4:junit4] 2> 1083165 T4499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1083165 T4499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data
[junit4:junit4] 2> 1083165 T4499 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data/index/
[junit4:junit4] 2> 1083166 T4499 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1083166 T4499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data/index
[junit4:junit4] 2> 1083167 T4499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b47922 lockFactory=org.apache.lucene.store.NativeFSLockFactory@78c476),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1083167 T4499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1083169 T4499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1083169 T4499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1083170 T4499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1083170 T4499 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1083171 T4499 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1083171 T4499 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1083171 T4499 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1083171 T4499 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1083172 T4499 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1083174 T4499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1083176 T4499 oass.SolrIndexSearcher.<init> Opening Searcher@1dfdcd7 main
[junit4:junit4] 2> 1083177 T4499 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/control/data/tlog
[junit4:junit4] 2> 1083177 T4499 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1083178 T4499 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1083181 T4500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dfdcd7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1083183 T4499 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1083183 T4499 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50857 collection:control_collection shard:shard1
[junit4:junit4] 2> 1083183 T4499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1083188 T4499 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1083190 T4499 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1083190 T4499 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1083190 T4499 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50857/collection1/
[junit4:junit4] 2> 1083190 T4499 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1083190 T4499 oasc.SyncStrategy.syncToMe http://127.0.0.1:50857/collection1/ has no replicas
[junit4:junit4] 2> 1083191 T4499 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50857/collection1/
[junit4:junit4] 2> 1083191 T4499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1083647 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1083652 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1083696 T4499 oasc.ZkController.register We are http://127.0.0.1:50857/collection1/ and leader is http://127.0.0.1:50857/collection1/
[junit4:junit4] 2> 1083697 T4499 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50857
[junit4:junit4] 2> 1083697 T4499 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1083697 T4499 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1083697 T4499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1083699 T4499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1083700 T4474 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1083701 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1083701 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1083706 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1083707 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1083708 T4503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d372eb name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1083709 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1083710 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1083712 T4474 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1083782 T4474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1083783 T4474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34390
[junit4:junit4] 2> 1083784 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1083784 T4474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1083785 T4474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724
[junit4:junit4] 2> 1083785 T4474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/solr.xml
[junit4:junit4] 2> 1083786 T4474 oasc.CoreContainer.<init> New CoreContainer 23928616
[junit4:junit4] 2> 1083786 T4474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/'
[junit4:junit4] 2> 1083787 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/'
[junit4:junit4] 2> 1083850 T4474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1083850 T4474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1083851 T4474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1083851 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1083852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1083852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1083852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1083853 T4474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1083853 T4474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1083854 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1083867 T4474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1083868 T4474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39152/solr
[junit4:junit4] 2> 1083868 T4474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1083869 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1083870 T4514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c3430d name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1083871 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1083872 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1083876 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1083877 T4516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f1f34c name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1083877 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1083880 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1084883 T4474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34390_
[junit4:junit4] 2> 1084884 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34390_
[junit4:junit4] 2> 1084886 T4503 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1084886 T4496 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1084886 T4516 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1084886 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1084891 T4517 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1084891 T4517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1085156 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1085157 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50857__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50857_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50857"}
[junit4:junit4] 2> 1085160 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34390_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34390"}
[junit4:junit4] 2> 1085160 T4497 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1085160 T4497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1085165 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1085165 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1085165 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1085892 T4517 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/collection1
[junit4:junit4] 2> 1085893 T4517 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1085894 T4517 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1085894 T4517 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1085895 T4517 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/collection1/'
[junit4:junit4] 2> 1085896 T4517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/collection1/lib/README' to classloader
[junit4:junit4] 2> 1085896 T4517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1085924 T4517 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1085959 T4517 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1085960 T4517 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1085964 T4517 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1086348 T4517 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1086351 T4517 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1086352 T4517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1086355 T4517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1086374 T4517 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1086374 T4517 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1372063740724/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1/
[junit4:junit4] 2> 1086374 T4517 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@104cf88
[junit4:junit4] 2> 1086375 T4517 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1086375 T4517 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1
[junit4:junit4] 2> 1086375 T4517 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1/index/
[junit4:junit4] 2> 1086375 T4517 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1086376 T4517 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1/index
[junit4:junit4] 2> 1086377 T4517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1086377 T4517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1086379 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1086379 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1086379 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1086380 T4517 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1086381 T4517 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1086381 T4517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1086381 T4517 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1086382 T4517 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1086382 T4517 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1086385 T4517 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1086388 T4517 oass.SolrIndexSearcher.<init> Opening Searcher@565611 main
[junit4:junit4] 2> 1086389 T4517 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty1/tlog
[junit4:junit4] 2> 1086389 T4517 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1086389 T4517 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1086396 T4518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@565611 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1086399 T4517 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1086399 T4517 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34390 collection:collection1 shard:shard1
[junit4:junit4] 2> 1086400 T4517 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1086406 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1086408 T4517 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1086409 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1086409 T4517 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34390/collection1/
[junit4:junit4] 2> 1086409 T4517 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1086409 T4517 oasc.SyncStrategy.syncToMe http://127.0.0.1:34390/collection1/ has no replicas
[junit4:junit4] 2> 1086410 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34390/collection1/
[junit4:junit4] 2> 1086410 T4517 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1086670 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1086676 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1086676 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1086676 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1086718 T4517 oasc.ZkController.register We are http://127.0.0.1:34390/collection1/ and leader is http://127.0.0.1:34390/collection1/
[junit4:junit4] 2> 1086718 T4517 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34390
[junit4:junit4] 2> 1086719 T4517 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1086719 T4517 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1086719 T4517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1086721 T4517 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1086722 T4474 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1086722 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1086723 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1086796 T4474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1086798 T4474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43993
[junit4:junit4] 2> 1086799 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1086799 T4474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1086800 T4474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740
[junit4:junit4] 2> 1086800 T4474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/solr.xml
[junit4:junit4] 2> 1086801 T4474 oasc.CoreContainer.<init> New CoreContainer 9103704
[junit4:junit4] 2> 1086801 T4474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/'
[junit4:junit4] 2> 1086802 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/'
[junit4:junit4] 2> 1086865 T4474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1086865 T4474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1086866 T4474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1086866 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1086866 T4474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1086867 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1086867 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1086868 T4474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1086868 T4474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1086869 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1086872 T4474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1086873 T4474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39152/solr
[junit4:junit4] 2> 1086874 T4474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1086874 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1086876 T4530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@128647d name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1086876 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1086877 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1086880 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1086881 T4532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16ad9d7 name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1086882 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1086884 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1087887 T4474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43993_
[junit4:junit4] 2> 1087890 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43993_
[junit4:junit4] 2> 1087892 T4516 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1087892 T4496 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1087892 T4532 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1087892 T4503 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1087892 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1087892 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1087893 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1087898 T4533 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1087898 T4533 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1088180 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1088181 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34390__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34390_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34390"}
[junit4:junit4] 2> 1088183 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43993_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43993"}
[junit4:junit4] 2> 1088184 T4497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1088184 T4497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1088186 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1088186 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1088186 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1088186 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1088899 T4533 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/collection1
[junit4:junit4] 2> 1088899 T4533 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1088900 T4533 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1088901 T4533 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1088902 T4533 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/collection1/'
[junit4:junit4] 2> 1088902 T4533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/collection1/lib/README' to classloader
[junit4:junit4] 2> 1088903 T4533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1088943 T4533 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1088978 T4533 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1088980 T4533 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1088983 T4533 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1089351 T4533 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1089353 T4533 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1089355 T4533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1089369 T4533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1089388 T4533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1089389 T4533 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1372063743740/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2/
[junit4:junit4] 2> 1089389 T4533 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@104cf88
[junit4:junit4] 2> 1089389 T4533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1089390 T4533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2
[junit4:junit4] 2> 1089390 T4533 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2/index/
[junit4:junit4] 2> 1089390 T4533 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1089391 T4533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2/index
[junit4:junit4] 2> 1089392 T4533 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1089392 T4533 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1089394 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1089394 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1089394 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1089395 T4533 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1089396 T4533 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1089396 T4533 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1089396 T4533 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1089397 T4533 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1089397 T4533 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1089399 T4533 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1089402 T4533 oass.SolrIndexSearcher.<init> Opening Searcher@1aef31f main
[junit4:junit4] 2> 1089402 T4533 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty2/tlog
[junit4:junit4] 2> 1089402 T4533 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1089403 T4533 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1089410 T4534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1aef31f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1089413 T4533 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1089414 T4533 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43993 collection:collection1 shard:shard2
[junit4:junit4] 2> 1089414 T4533 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1089419 T4533 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1089421 T4533 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1089422 T4533 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1089422 T4533 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43993/collection1/
[junit4:junit4] 2> 1089422 T4533 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1089422 T4533 oasc.SyncStrategy.syncToMe http://127.0.0.1:43993/collection1/ has no replicas
[junit4:junit4] 2> 1089423 T4533 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43993/collection1/
[junit4:junit4] 2> 1089423 T4533 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1089691 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1089696 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1089696 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1089696 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1089696 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1089729 T4533 oasc.ZkController.register We are http://127.0.0.1:43993/collection1/ and leader is http://127.0.0.1:43993/collection1/
[junit4:junit4] 2> 1089730 T4533 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43993
[junit4:junit4] 2> 1089730 T4533 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1089730 T4533 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1089731 T4533 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1089732 T4533 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1089733 T4474 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1089734 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1089734 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1089797 T4474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1089799 T4474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44860
[junit4:junit4] 2> 1089799 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1089800 T4474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1089800 T4474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749
[junit4:junit4] 2> 1089800 T4474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/solr.xml
[junit4:junit4] 2> 1089801 T4474 oasc.CoreContainer.<init> New CoreContainer 21320689
[junit4:junit4] 2> 1089801 T4474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/'
[junit4:junit4] 2> 1089801 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/'
[junit4:junit4] 2> 1089850 T4474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1089850 T4474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1089851 T4474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1089851 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1089851 T4474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1089852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1089852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1089852 T4474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1089853 T4474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1089853 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1089856 T4474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1089857 T4474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39152/solr
[junit4:junit4] 2> 1089857 T4474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1089858 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1089859 T4546 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e62d5d name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1089859 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1089860 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1089863 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1089864 T4548 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16e047a name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1089865 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1089868 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1090870 T4474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44860_
[junit4:junit4] 2> 1090871 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44860_
[junit4:junit4] 2> 1090873 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1090873 T4503 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1090874 T4516 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1090874 T4496 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1090874 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1090874 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1090874 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1090875 T4548 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1090875 T4532 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1090880 T4549 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1090880 T4549 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1091200 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1091201 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43993__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43993_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43993"}
[junit4:junit4] 2> 1091204 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44860_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44860"}
[junit4:junit4] 2> 1091205 T4497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1091205 T4497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1091207 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1091208 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1091207 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1091207 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1091207 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1091881 T4549 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/collection1
[junit4:junit4] 2> 1091881 T4549 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1091882 T4549 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1091882 T4549 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1091883 T4549 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/collection1/'
[junit4:junit4] 2> 1091884 T4549 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/collection1/lib/README' to classloader
[junit4:junit4] 2> 1091884 T4549 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1091926 T4549 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1091961 T4549 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1091963 T4549 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1091966 T4549 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1092334 T4549 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1092336 T4549 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1092338 T4549 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1092341 T4549 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1092359 T4549 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1092359 T4549 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1372063746749/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/
[junit4:junit4] 2> 1092359 T4549 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@104cf88
[junit4:junit4] 2> 1092360 T4549 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1092360 T4549 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3
[junit4:junit4] 2> 1092360 T4549 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index/
[junit4:junit4] 2> 1092360 T4549 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1092361 T4549 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index
[junit4:junit4] 2> 1092362 T4549 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e2402f lockFactory=org.apache.lucene.store.NativeFSLockFactory@c09a4f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1092362 T4549 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1092364 T4549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1092364 T4549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1092364 T4549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1092365 T4549 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1092366 T4549 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1092366 T4549 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1092366 T4549 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1092367 T4549 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1092367 T4549 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1092369 T4549 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1092383 T4549 oass.SolrIndexSearcher.<init> Opening Searcher@a264c8 main
[junit4:junit4] 2> 1092383 T4549 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/tlog
[junit4:junit4] 2> 1092384 T4549 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1092384 T4549 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1092389 T4550 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a264c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1092391 T4549 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1092391 T4549 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44860 collection:collection1 shard:shard1
[junit4:junit4] 2> 1092394 T4549 oasc.ZkController.register We are http://127.0.0.1:44860/collection1/ and leader is http://127.0.0.1:34390/collection1/
[junit4:junit4] 2> 1092394 T4549 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44860
[junit4:junit4] 2> 1092394 T4549 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1092394 T4549 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4031 name=collection1 org.apache.solr.core.SolrCore@2adeaa url=http://127.0.0.1:44860/collection1 node=127.0.0.1:44860_ C4031_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:44860_, base_url=http://127.0.0.1:44860}
[junit4:junit4] 2> 1092395 T4551 C4031 P44860 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1092395 T4549 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1092395 T4551 C4031 P44860 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1092395 T4551 C4031 P44860 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1092396 T4551 C4031 P44860 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1092397 T4474 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1092398 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1092397 T4551 C4031 P44860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1092399 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1092402 T4512 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:44860__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1092465 T4474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1092467 T4474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43848
[junit4:junit4] 2> 1092468 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1092468 T4474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1092468 T4474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415
[junit4:junit4] 2> 1092469 T4474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/solr.xml
[junit4:junit4] 2> 1092469 T4474 oasc.CoreContainer.<init> New CoreContainer 5871745
[junit4:junit4] 2> 1092470 T4474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/'
[junit4:junit4] 2> 1092470 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/'
[junit4:junit4] 2> 1092518 T4474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1092519 T4474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1092519 T4474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1092519 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1092520 T4474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1092520 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1092520 T4474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1092521 T4474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1092521 T4474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1092521 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1092524 T4474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1092525 T4474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39152/solr
[junit4:junit4] 2> 1092525 T4474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1092526 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1092527 T4563 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54abc0 name:ZooKeeperConnection Watcher:127.0.0.1:39152 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1092528 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1092529 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1092532 T4474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1092533 T4565 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dd4c name:ZooKeeperConnection Watcher:127.0.0.1:39152/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1092533 T4474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1092537 T4474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1092712 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1092713 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44860__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44860_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44860"}
[junit4:junit4] 2> 1092834 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1092835 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1092835 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1092835 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1092835 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1092835 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1093403 T4512 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:44860__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1093403 T4512 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:44860__collection1&state=recovering&nodeName=127.0.0.1:44860_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 1093540 T4474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43848_
[junit4:junit4] 2> 1093541 T4474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43848_
[junit4:junit4] 2> 1093543 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1093543 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1093544 T4503 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093544 T4548 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093544 T4516 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093545 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1093544 T4496 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093545 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1093544 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1093546 T4532 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093545 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1093548 T4565 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1093550 T4566 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1093551 T4566 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1094340 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1094341 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43848"}
[junit4:junit4] 2> 1094341 T4497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1094342 T4497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1094395 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094395 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094395 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094395 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094395 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094395 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1094552 T4566 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/collection1
[junit4:junit4] 2> 1094552 T4566 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1094553 T4566 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1094553 T4566 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1094554 T4566 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/collection1/'
[junit4:junit4] 2> 1094555 T4566 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/collection1/lib/README' to classloader
[junit4:junit4] 2> 1094555 T4566 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1094603 T4566 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1094638 T4566 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1094639 T4566 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1094643 T4566 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1095035 T4566 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1095037 T4566 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1095039 T4566 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1095042 T4566 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1095058 T4566 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1095059 T4566 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1372063749415/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/
[junit4:junit4] 2> 1095059 T4566 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@104cf88
[junit4:junit4] 2> 1095059 T4566 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1095060 T4566 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4
[junit4:junit4] 2> 1095060 T4566 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index/
[junit4:junit4] 2> 1095060 T4566 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1095061 T4566 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index
[junit4:junit4] 2> 1095062 T4566 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1095062 T4566 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1095064 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1095064 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1095064 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1095065 T4566 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1095066 T4566 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1095066 T4566 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1095066 T4566 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1095067 T4566 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1095067 T4566 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1095069 T4566 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1095072 T4566 oass.SolrIndexSearcher.<init> Opening Searcher@1d83a4 main
[junit4:junit4] 2> 1095072 T4566 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/tlog
[junit4:junit4] 2> 1095073 T4566 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1095073 T4566 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1095079 T4567 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d83a4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1095082 T4566 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1095082 T4566 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43848 collection:collection1 shard:shard2
[junit4:junit4] 2> 1095097 T4566 oasc.ZkController.register We are http://127.0.0.1:43848/collection1/ and leader is http://127.0.0.1:43993/collection1/
[junit4:junit4] 2> 1095097 T4566 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43848
[junit4:junit4] 2> 1095098 T4566 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1095098 T4566 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4032 name=collection1 org.apache.solr.core.SolrCore@125d49e url=http://127.0.0.1:43848/collection1 node=127.0.0.1:43848_ C4032_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:43848_, base_url=http://127.0.0.1:43848}
[junit4:junit4] 2> 1095098 T4568 C4032 P43848 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1095099 T4566 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1095099 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1095099 T4568 C4032 P43848 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1095099 T4568 C4032 P43848 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1095100 T4474 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1095100 T4474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1095100 T4474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1095101 T4568 C4032 P43848 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1095105 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1095106 T4527 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:43848__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1095106 T4474 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1095106 T4474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1095107 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C4033 name=collection1 org.apache.solr.core.SolrCore@2adeaa url=http://127.0.0.1:44860/collection1 node=127.0.0.1:44860_ C4033_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44860_, base_url=http://127.0.0.1:44860}
[junit4:junit4] 2> 1095404 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34390/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1095405 T4551 C4033 P44860 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44860 START replicas=[http://127.0.0.1:34390/collection1/] nUpdates=100
[junit4:junit4] 2> 1095405 T4551 C4033 P44860 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1095405 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1095405 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1095406 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1095406 T4551 C4033 P44860 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1095406 T4551 C4033 P44860 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34390/collection1/. core=collection1
[junit4:junit4] 2> 1095406 T4551 C4033 P44860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4034 name=collection1 org.apache.solr.core.SolrCore@b34026 url=http://127.0.0.1:34390/collection1 node=127.0.0.1:34390_ C4034_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34390_, base_url=http://127.0.0.1:34390, leader=true}
[junit4:junit4] 2> 1095412 T4509 C4034 P34390 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1095413 T4511 C4034 P34390 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1095414 T4511 C4034 P34390 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1095414 T4511 C4034 P34390 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1095415 T4511 C4034 P34390 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1095415 T4511 C4034 P34390 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1095416 T4511 C4034 P34390 oass.SolrIndexSearcher.<init> Opening Searcher@a3798a realtime
[junit4:junit4] 2> 1095416 T4511 C4034 P34390 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1095416 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4] 2> 1095417 T4551 C4033 P44860 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1095417 T4551 C4033 P44860 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1095419 T4510 C4034 P34390 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1095419 T4510 C4034 P34390 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1095419 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1095420 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1095420 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1095421 T4509 C4034 P34390 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1095422 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1095423 T4551 C4033 P44860 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index.20130624094912434
[junit4:junit4] 2> 1095423 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@175c451 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d918c8) fullCopy=false
[junit4:junit4] 2> 1095425 T4511 C4034 P34390 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1095426 T4551 C4033 P44860 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1095426 T4551 C4033 P44860 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1095426 T4551 C4033 P44860 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1095427 T4551 C4033 P44860 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e2402f lockFactory=org.apache.lucene.store.NativeFSLockFactory@c09a4f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e2402f lockFactory=org.apache.lucene.store.NativeFSLockFactory@c09a4f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1095427 T4551 C4033 P44860 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1095427 T4551 C4033 P44860 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1095428 T4551 C4033 P44860 oass.SolrIndexSearcher.<init> Opening Searcher@1be30f8 main
[junit4:junit4] 2> 1095429 T4550 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1be30f8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1095430 T4551 C4033 P44860 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index.20130624094912434 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index.20130624094912434;done=true>>]
[junit4:junit4] 2> 1095430 T4551 C4033 P44860 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index.20130624094912434
[junit4:junit4] 2> 1095430 T4551 C4033 P44860 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty3/index.20130624094912434
[junit4:junit4] 2> 1095431 T4551 C4033 P44860 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1095431 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1095431 T4551 C4033 P44860 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1095431 T4551 C4033 P44860 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1095432 T4551 C4033 P44860 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1095900 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1095901 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43848__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43848"}
[junit4:junit4] 2> 1095904 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44860__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44860_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44860"}
[junit4:junit4] 2> 1095964 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1095965 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1095965 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1095965 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1095965 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1095965 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1096106 T4527 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:43848__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1096107 T4527 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:43848__collection1&state=recovering&nodeName=127.0.0.1:43848_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 1096108 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1097110 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C4032_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:43848_, base_url=http://127.0.0.1:43848}
[junit4:junit4] 2> 1098107 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43993/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1098108 T4568 C4032 P43848 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43848 START replicas=[http://127.0.0.1:43993/collection1/] nUpdates=100
[junit4:junit4] 2> 1098109 T4568 C4032 P43848 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1098109 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1098109 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1098109 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1098109 T4568 C4032 P43848 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1098110 T4568 C4032 P43848 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43993/collection1/. core=collection1
[junit4:junit4] 2> 1098110 T4568 C4032 P43848 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1098111 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C4035 name=collection1 org.apache.solr.core.SolrCore@140a2b0 url=http://127.0.0.1:43993/collection1 node=127.0.0.1:43993_ C4035_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43993_, base_url=http://127.0.0.1:43993, leader=true}
[junit4:junit4] 2> 1098113 T4527 C4035 P43993 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1098114 T4525 C4035 P43993 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1098116 T4525 C4035 P43993 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1098116 T4525 C4035 P43993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1098117 T4525 C4035 P43993 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1098117 T4525 C4035 P43993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1098118 T4525 C4035 P43993 oass.SolrIndexSearcher.<init> Opening Searcher@3cf4ab realtime
[junit4:junit4] 2> 1098118 T4525 C4035 P43993 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1098118 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 1098119 T4568 C4032 P43848 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1098119 T4568 C4032 P43848 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1098121 T4528 C4035 P43993 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1098122 T4528 C4035 P43993 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1098122 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1098122 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1098123 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1098124 T4527 C4035 P43993 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1098125 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1098126 T4568 C4032 P43848 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index.20130624094915137
[junit4:junit4] 2> 1098126 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@128b836 lockFactory=org.apache.lucene.store.NativeFSLockFactory@129f2f1) fullCopy=false
[junit4:junit4] 2> 1098128 T4525 C4035 P43993 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1098130 T4568 C4032 P43848 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1098130 T4568 C4032 P43848 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1098131 T4568 C4032 P43848 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1098132 T4568 C4032 P43848 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1098132 T4568 C4032 P43848 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1098132 T4568 C4032 P43848 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1098132 T4568 C4032 P43848 oass.SolrIndexSearcher.<init> Opening Searcher@1dd4c35 main
[junit4:junit4] 2> 1098134 T4567 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dd4c35 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1098134 T4568 C4032 P43848 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index.20130624094915137 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index.20130624094915137;done=true>>]
[junit4:junit4] 2> 1098135 T4568 C4032 P43848 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index.20130624094915137
[junit4:junit4] 2> 1098135 T4568 C4032 P43848 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index.20130624094915137
[junit4:junit4] 2> 1098135 T4568 C4032 P43848 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1098135 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1098136 T4568 C4032 P43848 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1098136 T4568 C4032 P43848 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1098138 T4568 C4032 P43848 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1098973 T4497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1098974 T4497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43848__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43848_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43848"}
[junit4:junit4] 2> 1098977 T4503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1098977 T4532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1098978 T4496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1098977 T4516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1098977 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1098977 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1099113 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1099114 T4474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C4036 name=collection1 org.apache.solr.core.SolrCore@183e1dd url=http://127.0.0.1:50857/collection1 node=127.0.0.1:50857_ C4036_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:50857_, base_url=http://127.0.0.1:50857, leader=true}
[junit4:junit4] 2> 1099121 T4491 C4036 P50857 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1099123 T4491 C4036 P50857 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b47922 lockFactory=org.apache.lucene.store.NativeFSLockFactory@78c476),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1099123 T4491 C4036 P50857 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1099124 T4491 C4036 P50857 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b47922 lockFactory=org.apache.lucene.store.NativeFSLockFactory@78c476),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b47922 lockFactory=org.apache.lucene.store.NativeFSLockFactory@78c476),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1099124 T4491 C4036 P50857 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1099125 T4491 C4036 P50857 oass.SolrIndexSearcher.<init> Opening Searcher@9c2bbf main
[junit4:junit4] 2> 1099125 T4491 C4036 P50857 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1099127 T4500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9c2bbf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1099127 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1099131 T4511 C4034 P34390 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34390/collection1/, StdNode: http://127.0.0.1:44860/collection1/, StdNode: http://127.0.0.1:43993/collection1/, StdNode: http://127.0.0.1:43848/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1099135 T4525 C4035 P43993 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1099136 T4525 C4035 P43993 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1511bca lockFactory=org.apache.lucene.store.NativeFSLockFactory@2717d4),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1099136 T4525 C4035 P43993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1099135 T4512 C4034 P34390 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1099137 T4525 C4035 P43993 oass.SolrIndexSearcher.<init> Opening Searcher@3720ba main
[junit4:junit4] 2> 1099138 T4525 C4035 P43993 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1099139 T4534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3720ba main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> C4032_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43848_, base_url=http://127.0.0.1:43848}
[junit4:junit4] 2> 1099140 T4561 C4032 P43848 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1099140 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1099140 T4561 C4032 P43848 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1099141 T4561 C4032 P43848 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1099141 T4512 C4034 P34390 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1799462 lockFactory=org.apache.lucene.store.NativeFSLockFactory@125522f),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1099143 T4512 C4034 P34390 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1099142 T4561 C4032 P43848 oass.SolrIndexSearcher.<init> Opening Searcher@97a38f main
[junit4:junit4] 2> 1099143 T4512 C4034 P34390 oass.SolrIndexSearcher.<init> Opening Searcher@368f9e main
[junit4:junit4] 2> C4033_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44860_, base_url=http://127.0.0.1:44860}
[junit4:junit4] 2> 1099143 T4542 C4033 P44860 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1099144 T4567 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@97a38f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1099145 T4542 C4033 P44860 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e2402f lockFactory=org.apache.lucene.store.NativeFSLockFactory@c09a4f),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e2402f lockFactory=org.apache.lucene.store.NativeFSLockFactory@c09a4f),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1099145 T4542 C4033 P44860 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1099144 T4512 C4034 P34390 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1099144 T4561 C4032 P43848 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1099146 T4542 C4033 P44860 oass.SolrIndexSearcher.<init> Opening Searcher@12f25c6 main
[junit4:junit4] 2> 1099145 T4518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@368f9e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1099147 T4542 C4033 P44860 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1099146 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1099148 T4550 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12f25c6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1099148 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1099149 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1099149 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1099150 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1099152 T4510 C4034 P34390 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1099154 T4544 C4033 P44860 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1099155 T4528 C4035 P43993 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1099156 T4559 C4032 P43848 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1101160 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438713127287914496)} 0 1
[junit4:junit4] 2> 1101164 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438713127290011648&update.from=http://127.0.0.1:43993/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438713127290011648)} 0 0
[junit4:junit4] 2> 1101166 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438713127293157376&update.from=http://127.0.0.1:34390/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438713127293157376)} 0 0
[junit4:junit4] 2> 1101167 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438713127293157376)} 0 3
[junit4:junit4] 2> 1101167 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438713127290011648)} 0 6
[junit4:junit4] 2> 1101169 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0 (1438713127298400256)]} 0 0
[junit4:junit4] 2> 1101174 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!0 (1438713127301545984)]} 0 0
[junit4:junit4] 2> 1101175 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!0 (1438713127301545984)]} 0 3
[junit4:junit4] 2> 1101176 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0]} 0 5
[junit4:junit4] 2> 1101178 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1 (1438713127307837440)]} 0 0
[junit4:junit4] 2> 1101183 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!1 (1438713127310983168)]} 0 0
[junit4:junit4] 2> 1101184 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!1 (1438713127310983168)]} 0 3
[junit4:junit4] 2> 1101185 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1]} 0 5
[junit4:junit4] 2> 1101187 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2 (1438713127317274624)]} 0 1
[junit4:junit4] 2> 1101191 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!2 (1438713127319371776)]} 0 1
[junit4:junit4] 2> 1101191 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!2 (1438713127319371776)]} 0 2
[junit4:junit4] 2> 1101192 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2]} 0 4
[junit4:junit4] 2> 1101194 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3 (1438713127324614656)]} 0 0
[junit4:junit4] 2> 1101200 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!3 (1438713127328808960)]} 0 1
[junit4:junit4] 2> 1101200 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!3 (1438713127328808960)]} 0 2
[junit4:junit4] 2> 1101201 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3]} 0 5
[junit4:junit4] 2> 1101203 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4 (1438713127334051840)]} 0 0
[junit4:junit4] 2> 1101207 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!4 (1438713127336148992)]} 0 0
[junit4:junit4] 2> 1101208 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4 (1438713127336148992)]} 0 3
[junit4:junit4] 2> 1101210 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5 (1438713127341391872)]} 0 0
[junit4:junit4] 2> 1101216 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!5 (1438713127345586176)]} 0 1
[junit4:junit4] 2> 1101217 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!5 (1438713127345586176)]} 0 3
[junit4:junit4] 2> 1101217 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5]} 0 5
[junit4:junit4] 2> 1101219 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438713127350829056)]} 0 0
[junit4:junit4] 2> 1101223 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!6 (1438713127352926208)]} 0 0
[junit4:junit4] 2> 1101224 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438713127352926208)]} 0 3
[junit4:junit4] 2> 1101226 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7 (1438713127358169088)]} 0 0
[junit4:junit4] 2> 1101230 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!7 (1438713127360266240)]} 0 0
[junit4:junit4] 2> 1101231 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7 (1438713127360266240)]} 0 3
[junit4:junit4] 2> 1101233 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8 (1438713127365509120)]} 0 0
[junit4:junit4] 2> 1101239 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!8 (1438713127369703424)]} 0 1
[junit4:junit4] 2> 1101239 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!8 (1438713127369703424)]} 0 2
[junit4:junit4] 2> 1101240 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8]} 0 5
[junit4:junit4] 2> 1101242 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9 (1438713127374946304)]} 0 1
[junit4:junit4] 2> 1101247 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!9 (1438713127378092032)]} 0 1
[junit4:junit4] 2> 1101247 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!9 (1438713127378092032)]} 0 2
[junit4:junit4] 2> 1101248 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9]} 0 4
[junit4:junit4] 2> 1101250 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10 (1438713127383334912)]} 0 0
[junit4:junit4] 2> 1101255 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!10 (1438713127386480640)]} 0 0
[junit4:junit4] 2> 1101256 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!10 (1438713127386480640)]} 0 3
[junit4:junit4] 2> 1101257 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10]} 0 5
[junit4:junit4] 2> 1101259 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11 (1438713127391723520)]} 0 1
[junit4:junit4] 2> 1101264 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!11 (1438713127395917824)]} 0 1
[junit4:junit4] 2> 1101264 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!11 (1438713127395917824)]} 0 2
[junit4:junit4] 2> 1101265 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11]} 0 4
[junit4:junit4] 2> 1101267 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12 (1438713127401160704)]} 0 0
[junit4:junit4] 2> 1101273 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!12 (1438713127405355008)]} 0 0
[junit4:junit4] 2> 1101274 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!12 (1438713127405355008)]} 0 3
[junit4:junit4] 2> 1101274 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12]} 0 5
[junit4:junit4] 2> 1101277 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13 (1438713127410597888)]} 0 1
[junit4:junit4] 2> 1101283 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!13 (1438713127415840768)]} 0 1
[junit4:junit4] 2> 1101283 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!13 (1438713127415840768)]} 0 3
[junit4:junit4] 2> 1101284 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13]} 0 5
[junit4:junit4] 2> 1101286 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14 (1438713127421083648)]} 0 0
[junit4:junit4] 2> 1101292 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!14 (1438713127425277952)]} 0 0
[junit4:junit4] 2> 1101293 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!14 (1438713127425277952)]} 0 3
[junit4:junit4] 2> 1101293 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14]} 0 5
[junit4:junit4] 2> 1101295 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438713127430520832)]} 0 0
[junit4:junit4] 2> 1101299 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!15 (1438713127432617984)]} 0 0
[junit4:junit4] 2> 1101300 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438713127432617984)]} 0 3
[junit4:junit4] 2> 1101302 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16 (1438713127437860864)]} 0 0
[junit4:junit4] 2> 1101308 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!16 (1438713127442055168)]} 0 1
[junit4:junit4] 2> 1101308 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!16 (1438713127442055168)]} 0 2
[junit4:junit4] 2> 1101309 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16]} 0 5
[junit4:junit4] 2> 1101311 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17 (1438713127447298048)]} 0 0
[junit4:junit4] 2> 1101315 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!17 (1438713127449395200)]} 0 0
[junit4:junit4] 2> 1101316 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17 (1438713127449395200)]} 0 3
[junit4:junit4] 2> 1101318 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438713127454638080)]} 0 0
[junit4:junit4] 2> 1101322 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!18 (1438713127456735232)]} 0 0
[junit4:junit4] 2> 1101323 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438713127456735232)]} 0 3
[junit4:junit4] 2> 1101325 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19 (1438713127461978112)]} 0 0
[junit4:junit4] 2> 1101331 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!19 (1438713127466172416)]} 0 1
[junit4:junit4] 2> 1101332 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!19 (1438713127466172416)]} 0 3
[junit4:junit4] 2> 1101332 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19]} 0 5
[junit4:junit4] 2> 1101334 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20 (1438713127471415296)]} 0 0
[junit4:junit4] 2> 1101340 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!20 (1438713127475609600)]} 0 0
[junit4:junit4] 2> 1101341 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!20 (1438713127475609600)]} 0 3
[junit4:junit4] 2> 1101341 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20]} 0 5
[junit4:junit4] 2> 1101344 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21 (1438713127480852480)]} 0 1
[junit4:junit4] 2> 1101349 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!21 (1438713127485046784)]} 0 0
[junit4:junit4] 2> 1101350 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!21 (1438713127485046784)]} 0 3
[junit4:junit4] 2> 1101350 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21]} 0 4
[junit4:junit4] 2> 1101353 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438713127490289664)]} 0 1
[junit4:junit4] 2> 1101356 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!22 (1438713127493435392)]} 0 0
[junit4:junit4] 2> 1101357 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438713127493435392)]} 0 2
[junit4:junit4] 2> 1101359 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23 (1438713127497629696)]} 0 0
[junit4:junit4] 2> 1101365 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!23 (1438713127501824000)]} 0 1
[junit4:junit4] 2> 1101365 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!23 (1438713127501824000)]} 0 2
[junit4:junit4] 2> 1101366 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23]} 0 5
[junit4:junit4] 2> 1101368 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438713127507066880)]} 0 0
[junit4:junit4] 2> 1101372 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!24 (1438713127509164032)]} 0 0
[junit4:junit4] 2> 1101373 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438713127509164032)]} 0 3
[junit4:junit4] 2> 1101375 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438713127514406912)]} 0 0
[junit4:junit4] 2> 1101379 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!25 (1438713127516504064)]} 0 0
[junit4:junit4] 2> 1101380 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438713127516504064)]} 0 3
[junit4:junit4] 2> 1101383 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26 (1438713127522795520)]} 0 1
[junit4:junit4] 2> 1101387 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!26 (1438713127524892672)]} 0 0
[junit4:junit4] 2> 1101388 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26 (1438713127524892672)]} 0 3
[junit4:junit4] 2> 1101390 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27 (1438713127530135552)]} 0 0
[junit4:junit4] 2> 1101396 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!27 (1438713127534329856)]} 0 1
[junit4:junit4] 2> 1101397 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!27 (1438713127534329856)]} 0 3
[junit4:junit4] 2> 1101397 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27]} 0 5
[junit4:junit4] 2> 1101399 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438713127539572736)]} 0 0
[junit4:junit4] 2> 1101405 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!28 (1438713127543767040)]} 0 1
[junit4:junit4] 2> 1101406 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!28 (1438713127543767040)]} 0 3
[junit4:junit4] 2> 1101406 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28]} 0 5
[junit4:junit4] 2> 1101408 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438713127549009920)]} 0 0
[junit4:junit4] 2> 1101412 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!29 (1438713127551107072)]} 0 0
[junit4:junit4] 2> 1101413 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438713127551107072)]} 0 3
[junit4:junit4] 2> 1101415 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30 (1438713127556349952)]} 0 0
[junit4:junit4] 2> 1101421 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!30 (1438713127560544256)]} 0 0
[junit4:junit4] 2> 1101422 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!30 (1438713127560544256)]} 0 3
[junit4:junit4] 2> 1101422 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30]} 0 5
[junit4:junit4] 2> 1101425 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31 (1438713127565787136)]} 0 1
[junit4:junit4] 2> 1101430 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!31 (1438713127569981440)]} 0 0
[junit4:junit4] 2> 1101431 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!31 (1438713127569981440)]} 0 3
[junit4:junit4] 2> 1101431 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31]} 0 4
[junit4:junit4] 2> 1101434 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32 (1438713127575224320)]} 0 1
[junit4:junit4] 2> 1101439 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!32 (1438713127579418624)]} 0 0
[junit4:junit4] 2> 1101440 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!32 (1438713127579418624)]} 0 3
[junit4:junit4] 2> 1101441 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32]} 0 5
[junit4:junit4] 2> 1101443 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33 (1438713127585710080)]} 0 0
[junit4:junit4] 2> 1101449 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!33 (1438713127589904384)]} 0 1
[junit4:junit4] 2> 1101449 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!33 (1438713127589904384)]} 0 2
[junit4:junit4] 2> 1101450 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33]} 0 5
[junit4:junit4] 2> 1101452 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34 (1438713127595147264)]} 0 0
[junit4:junit4] 2> 1101458 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!34 (1438713127599341568)]} 0 0
[junit4:junit4] 2> 1101459 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!34 (1438713127599341568)]} 0 3
[junit4:junit4] 2> 1101459 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34]} 0 5
[junit4:junit4] 2> 1101462 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438713127604584448)]} 0 1
[junit4:junit4] 2> 1101467 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!35 (1438713127608778752)]} 0 0
[junit4:junit4] 2> 1101468 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!35 (1438713127608778752)]} 0 3
[junit4:junit4] 2> 1101468 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35]} 0 4
[junit4:junit4] 2> 1101471 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36 (1438713127614021632)]} 0 1
[junit4:junit4] 2> 1101476 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!36 (1438713127618215936)]} 0 0
[junit4:junit4] 2> 1101476 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!36 (1438713127618215936)]} 0 2
[junit4:junit4] 2> 1101477 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36]} 0 5
[junit4:junit4] 2> 1101479 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37 (1438713127623458816)]} 0 0
[junit4:junit4] 2> 1101485 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!37 (1438713127627653120)]} 0 1
[junit4:junit4] 2> 1101486 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!37 (1438713127627653120)]} 0 3
[junit4:junit4] 2> 1101486 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37]} 0 5
[junit4:junit4] 2> 1101488 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438713127632896000)]} 0 0
[junit4:junit4] 2> 1101494 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!38 (1438713127637090304)]} 0 0
[junit4:junit4] 2> 1101495 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!38 (1438713127637090304)]} 0 3
[junit4:junit4] 2> 1101495 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38]} 0 5
[junit4:junit4] 2> 1101497 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39 (1438713127642333184)]} 0 0
[junit4:junit4] 2> 1101503 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!39 (1438713127646527488)]} 0 0
[junit4:junit4] 2> 1101504 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!39 (1438713127646527488)]} 0 3
[junit4:junit4] 2> 1101504 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39]} 0 4
[junit4:junit4] 2> 1101506 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40 (1438713127651770368)]} 0 0
[junit4:junit4] 2> 1101512 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!40 (1438713127655964672)]} 0 0
[junit4:junit4] 2> 1101513 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!40 (1438713127655964672)]} 0 3
[junit4:junit4] 2> 1101513 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40]} 0 5
[junit4:junit4] 2> 1101515 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41 (1438713127661207552)]} 0 0
[junit4:junit4] 2> 1101521 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!41 (1438713127665401856)]} 0 0
[junit4:junit4] 2> 1101521 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!41 (1438713127665401856)]} 0 2
[junit4:junit4] 2> 1101522 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41]} 0 5
[junit4:junit4] 2> 1101524 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42 (1438713127670644736)]} 0 0
[junit4:junit4] 2> 1101530 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!42 (1438713127674839040)]} 0 0
[junit4:junit4] 2> 1101531 T4511 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!42 (1438713127674839040)]} 0 3
[junit4:junit4] 2> 1101531 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42]} 0 5
[junit4:junit4] 2> 1101534 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43 (1438713127680081920)]} 0 1
[junit4:junit4] 2> 1101539 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!43 (1438713127684276224)]} 0 0
[junit4:junit4] 2> 1101540 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!43 (1438713127684276224)]} 0 3
[junit4:junit4] 2> 1101540 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43]} 0 4
[junit4:junit4] 2> 1101543 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44 (1438713127690567680)]} 0 0
[junit4:junit4] 2> 1101548 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!44 (1438713127693713408)]} 0 0
[junit4:junit4] 2> 1101549 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[s!44 (1438713127693713408)]} 0 3
[junit4:junit4] 2> 1101549 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44]} 0 4
[junit4:junit4] 2> 1101551 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45 (1438713127698956288)]} 0 0
[junit4:junit4] 2> 1101557 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!45 (1438713127703150592)]} 0 0
[junit4:junit4] 2> 1101558 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!45 (1438713127703150592)]} 0 3
[junit4:junit4] 2> 1101558 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45]} 0 5
[junit4:junit4] 2> 1101560 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46 (1438713127708393472)]} 0 0
[junit4:junit4] 2> 1101566 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!46 (1438713127712587776)]} 0 0
[junit4:junit4] 2> 1101567 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!46 (1438713127712587776)]} 0 3
[junit4:junit4] 2> 1101568 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46]} 0 5
[junit4:junit4] 2> 1101570 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47 (1438713127718879232)]} 0 0
[junit4:junit4] 2> 1101576 T4542 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!47 (1438713127722024960)]} 0 1
[junit4:junit4] 2> 1101576 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!47 (1438713127722024960)]} 0 3
[junit4:junit4] 2> 1101577 T4558 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47]} 0 5
[junit4:junit4] 2> 1101579 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48 (1438713127728316416)]} 0 0
[junit4:junit4] 2> 1101585 T4541 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!48 (1438713127732510720)]} 0 1
[junit4:junit4] 2> 1101585 T4510 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!48 (1438713127732510720)]} 0 2
[junit4:junit4] 2> 1101586 T4527 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48]} 0 5
[junit4:junit4] 2> 1101588 T4491 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49 (1438713127737753600)]} 0 0
[junit4:junit4] 2> 1101593 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!49 (1438713127740899328)]} 0 0
[junit4:junit4] 2> 1101594 T4525 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!49 (1438713127740899328)]} 0 3
[junit4:junit4] 2> 1101594 T4544 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49]} 0 4
[junit4:junit4] 2> 1101597 T4489 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438713127746142208)]} 0 1
[junit4:junit4] 2> 1101602 T4559 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!50 (1438713127750336512)]} 0 0
[junit4:junit4] 2> 1101603 T4528 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[y!50 (1438713127750336512)]} 0 3
[junit4:junit4] 2> 1101604 T4509 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50]} 0 5
[junit4:junit4] 2> 1101606 T4492 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51 (1438713127756627968)]} 0 0
[junit4:junit4] 2> 1101611 T4543 C4033 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!51 (1438713127759773696)]} 0 0
[junit4:junit4] 2> 1101612 T4512 C4034 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[z!51 (1438713127759773696)]} 0 3
[junit4:junit4] 2> 1101612 T4561 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51]} 0 4
[junit4:junit4] 2> 1101615 T4490 C4036 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52 (1438713127765016576)]} 0 1
[junit4:junit4] 2> 1101620 T4560 C4032 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!52 (1438713127769210880)]} 0 0
[junit4:junit4] 2> 1101621 T4526 C4035 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!52 (1438713127769210880)]} 0 3
[junit4:junit4] 2> ASYNC NEW_CORE C4037 name=collection1 org.apache.solr.core.SolrCore@125d49e url=http://127.0.0.1:43848/collection1 node=127.0.0.1:43848_ C4037_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43848_, base_url=http://127.0.0.1:43848}
[junit4:junit4] 2> 1101640 T4558 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52]} 0 24
[junit4:junit4] 2> ASYNC NEW_CORE C4038 name=collection1 org.apache.solr.core.SolrCore@183e1dd url=http://127.0.0.1:50857/collection1 node=127.0.0.1:50857_ C4038_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:50857_, base_url=http://127.0.0.1:50857, leader=true}
[junit4:junit4] 2> 1101643 T4491 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53 (1438713127795425280)]} 0 0
[junit4:junit4] 2> ASYNC NEW_CORE C4039 name=collection1 org.apache.solr.core.SolrCore@2adeaa url=http://127.0.0.1:44860/collection1 node=127.0.0.1:44860_ C4039_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44860_, base_url=http://127.0.0.1:44860}
[junit4:junit4] 2> 1101648 T4541 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!53 (1438713127798571008)]} 0 0
[junit4:junit4] 2> ASYNC NEW_CORE C4040 name=collection1 org.apache.solr.core.SolrCore@b34026 url=http://127.0.0.1:34390/collection1 node=127.0.0.1:34390_ C4040_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34390_, base_url=http://127.0.0.1:34390, leader=true}
[junit4:junit4] 2> 1101649 T4510 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!53 (1438713127798571008)]} 0 3
[junit4:junit4] 2> 1101650 T4542 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53]} 0 5
[junit4:junit4] 2> 1101652 T4489 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54 (1438713127804862464)]} 0 0
[junit4:junit4] 2> 1101658 T4544 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!54 (1438713127809056768)]} 0 1
[junit4:junit4] 2> 1101658 T4511 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!54 (1438713127809056768)]} 0 2
[junit4:junit4] 2> ASYNC NEW_CORE C4041 name=collection1 org.apache.solr.core.SolrCore@140a2b0 url=http://127.0.0.1:43993/collection1 node=127.0.0.1:43993_ C4041_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43993_, base_url=http://127.0.0.1:43993, leader=true}
[junit4:junit4] 2> 1101659 T4525 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54]} 0 5
[junit4:junit4] 2> 1101661 T4492 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55 (1438713127814299648)]} 0 0
[junit4:junit4] 2> 1101666 T4559 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!55 (1438713127817445376)]} 0 0
[junit4:junit4] 2> 1101667 T4528 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!55 (1438713127817445376)]} 0 3
[junit4:junit4] 2> 1101667 T4543 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55]} 0 4
[junit4:junit4] 2> 1101669 T4490 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56 (1438713127822688256)]} 0 0
[junit4:junit4] 2> 1101673 T4561 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!56 (1438713127824785408)]} 0 0
[junit4:junit4] 2> 1101674 T4527 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56 (1438713127824785408)]} 0 3
[junit4:junit4] 2> 1101676 T4491 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57 (1438713127830028288)]} 0 0
[junit4:junit4] 2> 1101681 T4558 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!57 (1438713127833174016)]} 0 0
[junit4:junit4] 2> 1101682 T4526 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!57 (1438713127833174016)]} 0 3
[junit4:junit4] 2> 1101683 T4560 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57]} 0 5
[junit4:junit4] 2> 1101685 T4489 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!58 (1438713127839465472)]} 0 0
[junit4:junit4] 2> 1101690 T4541 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!58 (1438713127842611200)]} 0 0
[junit4:junit4] 2> 1101691 T4509 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!58 (1438713127842611200)]} 0 3
[junit4:junit4] 2> 1101691 T4559 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!58]} 0 4
[junit4:junit4] 2> 1101693 T4492 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!59 (1438713127847854080)]} 0 0
[junit4:junit4] 2> 1101699 T4542 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!59 (1438713127852048384)]} 0 1
[junit4:junit4] 2> 1101699 T4512 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!59 (1438713127852048384)]} 0 2
[junit4:junit4] 2> 1101699 T4525 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!59]} 0 4
[junit4:junit4] 2> 1101702 T4490 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!60 (1438713127856242688)]} 0 1
[junit4:junit4] 2> 1101707 T4544 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!60 (1438713127860436992)]} 0 0
[junit4:junit4] 2> 1101708 T4510 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!60 (1438713127860436992)]} 0 3
[junit4:junit4] 2> 1101708 T4528 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!60]} 0 5
[junit4:junit4] 2> 1101710 T4491 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!61 (1438713127865679872)]} 0 0
[junit4:junit4] 2> 1101716 T4543 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!61 (1438713127869874176)]} 0 0
[junit4:junit4] 2> 1101717 T4511 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!61 (1438713127869874176)]} 0 3
[junit4:junit4] 2> 1101717 T4527 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!61]} 0 5
[junit4:junit4] 2> 1101720 T4489 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!62 (1438713127875117056)]} 0 1
[junit4:junit4] 2> 1101725 T4541 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!62 (1438713127879311360)]} 0 0
[junit4:junit4] 2> 1101726 T4509 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!62 (1438713127879311360)]} 0 3
[junit4:junit4] 2> 1101726 T4561 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!62]} 0 4
[junit4:junit4] 2> 1101728 T4492 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!63 (1438713127884554240)]} 0 0
[junit4:junit4] 2> 1101734 T4560 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!63 (1438713127888748544)]} 0 1
[junit4:junit4] 2> 1101734 T4526 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43848/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!63 (1438713127888748544)]} 0 2
[junit4:junit4] 2> 1101735 T4558 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!63]} 0 5
[junit4:junit4] 2> 1101737 T4490 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!64 (1438713127893991424)]} 0 0
[junit4:junit4] 2> 1101743 T4559 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43993/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!64 (1438713127898185728)]} 0 0
[junit4:junit4] 2> 1101744 T4525 C4041 P43993 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!64 (1438713127898185728)]} 0 3
[junit4:junit4] 2> 1101744 T4512 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!64]} 0 5
[junit4:junit4] 2> 1101746 T4491 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!65 (1438713127903428608)]} 0 0
[junit4:junit4] 2> 1101751 T4544 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34390/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!65 (1438713127906574336)]} 0 0
[junit4:junit4] 2> 1101752 T4510 C4040 P34390 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44860/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!65 (1438713127906574336)]} 0 3
[junit4:junit4] 2> 1101752 T4542 C4039 P44860 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!65]} 0 4
[junit4:junit4] 2> 1101754 T4489 C4038 P50857 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!66 (1438713127911817216)]} 0 0
[junit4:junit4] 2> 1101760 T4561 C4037 P43848 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update pa
[...truncated too long message...]
t4] 2> 1291871 T4474 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1291871 T4474 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> ASYNC NEW_CORE C4183 name=collection1 org.apache.solr.core.SolrCore@125d49e url=http://127.0.0.1:43848/collection1 node=127.0.0.1:43848_ C4183_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43848_, base_url=http://127.0.0.1:43848, leader=true}
[junit4:junit4] 2> 1291878 T4474 C4183 P43848 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d76aca lockFactory=org.apache.lucene.store.NativeFSLockFactory@1353cc1),segFN=segments_5,generation=5}
[junit4:junit4] 2> 1291878 T4474 C4183 P43848 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 1291879 T4474 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1291879 T4474 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1291880 T4474 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4;done=false>>]
[junit4:junit4] 2> 1291880 T4474 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4
[junit4:junit4] 2> 1291880 T4474 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index;done=false>>]
[junit4:junit4] 2> 1291881 T4474 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372063737327/jetty4/index
[junit4:junit4] 2> 1291882 T4605 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89919569089593356-127.0.0.1:43848_-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 1291883 T4565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1291883 T4565 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1291883 T4565 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1291904 T4474 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 1291959 T4474 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1291961 T4474 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39152 39152
[junit4:junit4] 2> 1291962 T4474 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 50857
[junit4:junit4] 2> !!!! WARNING: best effort to remove /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-1372063737327 FAILED !!!!!
[junit4:junit4] 2> 1291962 T4474 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1291963 T4474 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39152 39152
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=7785BBA5FA80CFCF -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv_LV -Dtests.timezone=Europe/Isle_of_Man -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 212s J0 | ShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:34390 returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([7785BBA5FA80CFCF:F66335BD8DDFAFF3]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 1291994 T4474 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 211687 T4473 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 1292095 T4548 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1292095 T4548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1292095 T4548 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=lv_LV, timezone=Europe/Isle_of_Man
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Oracle Corporation 1.7.0_25 (32-bit)/cpus=8,threads=3,free=115577272,total=214097920
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestSolrDeletionPolicy2, QueryParsingTest, DistributedQueryElevationComponentTest, TestTrie, ResourceLoaderTest, ShowFileRequestHandlerTest, AutoCommitTest, OverseerCollectionProcessorTest, LegacyHTMLStripCharFilterTest, StatelessScriptUpdateProcessorFactoryTest, TestLRUCache, SchemaVersionSpecificBehaviorTest, TestUniqueKeyFieldResource, SuggesterFSTTest, CoreAdminHandlerTest, TestFieldCollectionResource, ZkCLITest, ShardRoutingTest, TestLMDirichletSimilarityFactory, TestReplicationHandler, SoftAutoCommitTest, TestCoreContainer, SOLR749Test, AlternateDirectoryTest, UpdateRequestProcessorFactoryTest, UnloadDistributedZkTest, IndexSchemaTest, BasicDistributedZkTest, TestConfig, FileBasedSpellCheckerTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, TestCharFilters, SystemInfoHandlerTest, MoreLikeThisHandlerTest, SliceStateTest, ResponseLogComponentTest, SearchHandlerTest, TestJoin, LeaderElectionTest, SyncSliceTest, ClusterStateUpdateTest, TestSolrXmlPersistence, PrimitiveFieldTypeTest, OverseerTest, BasicFunctionalityTest, TestPHPSerializedResponseWriter, TestBadConfig, PreAnalyzedFieldTest, TestFaceting, JsonLoaderTest, SolrIndexSplitterTest, CircularListTest, RAMDirectoryFactoryTest, PeerSyncTest, TestClassNameShortening, PrimUtilsTest, WordBreakSolrSpellCheckerTest, TestManagedSchema, MinimalSchemaTest, CachingDirectoryFactoryTest, SynonymTokenizerTest, TestCloudManagedSchema, TestFastWriter, QueryElevationComponentTest, DocumentBuilderTest, SolrInfoMBeanTest, TestSolrDeletionPolicy1, RecoveryZkTest, TestRemoteStreaming, UUIDFieldTest, CollectionsAPIDistributedZkTest, SliceStateUpdateTest, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TestStressReorder, HardAutoCommitTest, DistributedTermsComponentTest, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, TestIndexSearcher, HighlighterTest, CurrencyFieldOpenExchangeTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, TestCSVLoader, PolyFieldTest, TestPseudoReturnFields, QueryEqualityTest, SortByFunctionTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, TermsComponentTest, TestValueSourceCache, TermVectorComponentTest, FastVectorHighlighterTest, RegexBoostProcessorTest, ReturnFieldsTest, MBeansHandlerTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, TestLFUCache, TestPropInject, TestPropInjectDefaults, TestSolrIndexConfig, SolrIndexConfigTest, TestStressRecovery, ShardSplitTest]
[junit4:junit4] Completed on J0 in 211.96s, 1 test, 1 error <<< FAILURES!
[...truncated 10 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:366: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1243: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 error, 13 ignored (7 assumptions)
Total time: 39 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_25 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_25) - Build #
6286 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6286/
Java: 64bit/jdk1.7.0_25 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:60141/mj/e/collection1lastClient and got 252 from http://127.0.0.1:34628/mj/e/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:60141/mj/e/collection1lastClient and got 252 from http://127.0.0.1:34628/mj/e/collection1
at __randomizedtesting.SeedInfo.seed([15851157ED6233A5:94639F4F9A3D5399]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9249 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 55530 T259 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /mj/e
[junit4:junit4] 2> 55538 T259 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372068092747
[junit4:junit4] 2> 55539 T259 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 55540 T260 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 55640 T259 oasc.ZkTestServer.run start zk server on port:54516
[junit4:junit4] 2> 55642 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 55652 T266 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c495713 name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 55652 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 55652 T259 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 55657 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 55659 T268 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77647a8f name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 55659 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 55659 T259 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 55663 T259 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 55667 T259 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 55669 T259 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 55672 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 55672 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 55685 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 55686 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 55689 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 55690 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 55693 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 55693 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 55696 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 55697 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 55700 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 55700 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 55703 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 55703 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 55706 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 55707 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 55709 T259 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 55710 T259 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 55780 T259 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 55785 T259 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45893
[junit4:junit4] 2> 55786 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 55786 T259 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 55787 T259 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922
[junit4:junit4] 2> 55787 T259 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/solr.xml
[junit4:junit4] 2> 55788 T259 oasc.CoreContainer.<init> New CoreContainer 1841221836
[junit4:junit4] 2> 55788 T259 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/'
[junit4:junit4] 2> 55789 T259 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/'
[junit4:junit4] 2> 55862 T259 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 55863 T259 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 55863 T259 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 55864 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 55864 T259 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 55865 T259 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 55865 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 55866 T259 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 55866 T259 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 55867 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 55872 T259 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 55873 T259 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54516/solr
[junit4:junit4] 2> 55874 T259 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 55875 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 55877 T279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a30853b name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 55877 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 55880 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 55885 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 55887 T281 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41199ec2 name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 55888 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 55890 T259 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 55894 T259 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 55898 T259 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 55901 T259 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45893_mj%2Fe
[junit4:junit4] 2> 55902 T259 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45893_mj%2Fe
[junit4:junit4] 2> 55905 T259 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 55910 T259 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 55913 T259 oasc.Overseer.start Overseer (id=89919854526595075-127.0.0.1:45893_mj%2Fe-n_0000000000) starting
[junit4:junit4] 2> 55919 T259 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 55923 T283 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 55923 T259 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 55926 T259 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 55928 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 55931 T282 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 55935 T284 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 55935 T284 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 57436 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 57437 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45893_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45893/mj/e"}
[junit4:junit4] 2> 57437 T282 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 57437 T282 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 57442 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 57937 T284 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/collection1
[junit4:junit4] 2> 57938 T284 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 57939 T284 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 57939 T284 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 57941 T284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/collection1/'
[junit4:junit4] 2> 57942 T284 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/collection1/lib/README' to classloader
[junit4:junit4] 2> 57942 T284 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 57984 T284 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 58024 T284 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 58026 T284 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 58041 T284 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 58386 T284 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 58393 T284 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 58395 T284 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 58420 T284 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 58425 T284 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 58431 T284 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 58432 T284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 58433 T284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 58433 T284 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 58435 T284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 58435 T284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 58436 T284 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 58445 T284 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372068092922/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/
[junit4:junit4] 2> 58446 T284 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e7bc1f9
[junit4:junit4] 2> 58446 T284 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 58446 T284 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data
[junit4:junit4] 2> 58447 T284 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index/
[junit4:junit4] 2> 58447 T284 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 58447 T284 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index
[junit4:junit4] 2> 58451 T284 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 58451 T284 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 58453 T284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 58453 T284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 58454 T284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 58455 T284 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 58455 T284 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 58455 T284 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 58456 T284 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 58456 T284 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 58457 T284 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 58459 T284 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 58462 T284 oass.SolrIndexSearcher.<init> Opening Searcher@3c4d5ad7 main
[junit4:junit4] 2> 58463 T284 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 58463 T284 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 58474 T285 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3c4d5ad7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 58476 T284 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 58476 T284 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45893/mj/e collection:control_collection shard:shard1
[junit4:junit4] 2> 58477 T284 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 58484 T284 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 58487 T284 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 58487 T284 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 58488 T284 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45893/mj/e/collection1/
[junit4:junit4] 2> 58488 T284 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 58488 T284 oasc.SyncStrategy.syncToMe http://127.0.0.1:45893/mj/e/collection1/ has no replicas
[junit4:junit4] 2> 58488 T284 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45893/mj/e/collection1/
[junit4:junit4] 2> 58489 T284 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 58946 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 58955 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 58997 T284 oasc.ZkController.register We are http://127.0.0.1:45893/mj/e/collection1/ and leader is http://127.0.0.1:45893/mj/e/collection1/
[junit4:junit4] 2> 58997 T284 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45893/mj/e
[junit4:junit4] 2> 58998 T284 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 58998 T284 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 58998 T284 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 59000 T284 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 59002 T259 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 59002 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 59003 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 59009 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 59011 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 59013 T288 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27dcdbe name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 59013 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 59015 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 59017 T259 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 59109 T259 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 59111 T259 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60141
[junit4:junit4] 2> 59112 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 59113 T259 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 59113 T259 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227
[junit4:junit4] 2> 59113 T259 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/solr.xml
[junit4:junit4] 2> 59114 T259 oasc.CoreContainer.<init> New CoreContainer 383606531
[junit4:junit4] 2> 59114 T259 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/'
[junit4:junit4] 2> 59114 T259 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/'
[junit4:junit4] 2> 59156 T259 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 59156 T259 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 59157 T259 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 59157 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 59157 T259 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 59157 T259 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 59158 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 59158 T259 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 59158 T259 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 59159 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 59162 T259 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 59163 T259 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54516/solr
[junit4:junit4] 2> 59163 T259 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 59164 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 59166 T299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@777acbb name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 59166 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 59168 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 59172 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 59174 T301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@472bd646 name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 59174 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 59178 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 60182 T259 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60141_mj%2Fe
[junit4:junit4] 2> 60183 T259 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60141_mj%2Fe
[junit4:junit4] 2> 60186 T288 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 60187 T301 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 60187 T281 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 60187 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 60198 T302 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 60199 T302 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 60462 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 60463 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:45893_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45893_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45893/mj/e"}
[junit4:junit4] 2> 60466 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60141_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60141/mj/e"}
[junit4:junit4] 2> 60467 T282 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 60467 T282 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 60472 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 60472 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 60472 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 61200 T302 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/collection1
[junit4:junit4] 2> 61201 T302 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 61202 T302 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 61202 T302 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 61203 T302 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/collection1/'
[junit4:junit4] 2> 61204 T302 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/collection1/lib/README' to classloader
[junit4:junit4] 2> 61204 T302 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 61230 T302 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 61268 T302 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 61270 T302 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 61283 T302 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 61631 T302 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 61637 T302 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 61639 T302 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 61660 T302 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 61663 T302 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 61667 T302 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 61668 T302 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 61668 T302 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 61669 T302 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 61670 T302 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 61670 T302 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 61671 T302 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 61671 T302 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372068096227/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/
[junit4:junit4] 2> 61672 T302 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e7bc1f9
[junit4:junit4] 2> 61672 T302 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 61673 T302 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1
[junit4:junit4] 2> 61673 T302 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index/
[junit4:junit4] 2> 61674 T302 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 61674 T302 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index
[junit4:junit4] 2> 61679 T302 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 61679 T302 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 61682 T302 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 61683 T302 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 61683 T302 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 61684 T302 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 61685 T302 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 61685 T302 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 61685 T302 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 61686 T302 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 61687 T302 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 61689 T302 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 61697 T302 oass.SolrIndexSearcher.<init> Opening Searcher@8bc2356 main
[junit4:junit4] 2> 61697 T302 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 61698 T302 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 61704 T303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8bc2356 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 61707 T302 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 61708 T302 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60141/mj/e collection:collection1 shard:shard1
[junit4:junit4] 2> 61709 T302 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 61717 T302 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 61720 T302 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 61721 T302 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 61721 T302 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 61721 T302 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 61722 T302 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/ has no replicas
[junit4:junit4] 2> 61722 T302 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 61722 T302 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 61978 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 61987 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 61987 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 61987 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 62030 T302 oasc.ZkController.register We are http://127.0.0.1:60141/mj/e/collection1/ and leader is http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 62031 T302 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60141/mj/e
[junit4:junit4] 2> 62031 T302 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 62032 T302 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 62032 T302 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 62034 T302 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 62036 T259 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 62036 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 62037 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 62108 T259 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 62112 T259 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37678
[junit4:junit4] 2> 62113 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 62113 T259 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 62113 T259 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249
[junit4:junit4] 2> 62114 T259 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/solr.xml
[junit4:junit4] 2> 62114 T259 oasc.CoreContainer.<init> New CoreContainer 1483522142
[junit4:junit4] 2> 62115 T259 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/'
[junit4:junit4] 2> 62115 T259 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/'
[junit4:junit4] 2> 62156 T259 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 62156 T259 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 62157 T259 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 62157 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 62157 T259 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 62158 T259 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 62158 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 62158 T259 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 62159 T259 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 62159 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 62163 T259 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 62163 T259 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54516/solr
[junit4:junit4] 2> 62163 T259 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 62164 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 62166 T315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53a8db1b name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 62166 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 62168 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 62178 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 62180 T317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cd714bd name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 62180 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 62184 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 63188 T259 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37678_mj%2Fe
[junit4:junit4] 2> 63189 T259 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37678_mj%2Fe
[junit4:junit4] 2> 63191 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 63192 T301 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 63192 T281 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 63193 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 63192 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 63193 T317 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 63194 T288 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 63199 T318 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 63199 T318 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 63493 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 63495 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60141_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60141_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60141/mj/e"}
[junit4:junit4] 2> 63499 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37678_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37678/mj/e"}
[junit4:junit4] 2> 63499 T282 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 63500 T282 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 63504 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 63504 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 63504 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 63505 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 64200 T318 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/collection1
[junit4:junit4] 2> 64201 T318 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 64202 T318 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 64202 T318 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 64203 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/collection1/'
[junit4:junit4] 2> 64204 T318 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/collection1/lib/README' to classloader
[junit4:junit4] 2> 64204 T318 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 64231 T318 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 64267 T318 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 64269 T318 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 64277 T318 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 64596 T318 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 64602 T318 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 64605 T318 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 64627 T318 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 64635 T318 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 64639 T318 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 64640 T318 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 64641 T318 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 64641 T318 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 64642 T318 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 64643 T318 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 64643 T318 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 64643 T318 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372068099249/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/
[junit4:junit4] 2> 64644 T318 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e7bc1f9
[junit4:junit4] 2> 64645 T318 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 64645 T318 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2
[junit4:junit4] 2> 64646 T318 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index/
[junit4:junit4] 2> 64646 T318 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 64647 T318 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index
[junit4:junit4] 2> 64651 T318 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 64652 T318 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 64654 T318 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 64655 T318 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 64655 T318 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 64656 T318 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 64656 T318 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 64657 T318 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 64657 T318 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 64658 T318 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 64658 T318 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 64661 T318 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 64664 T318 oass.SolrIndexSearcher.<init> Opening Searcher@113a6ebb main
[junit4:junit4] 2> 64665 T318 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 64665 T318 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 64671 T319 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@113a6ebb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 64674 T318 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 64674 T318 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37678/mj/e collection:collection1 shard:shard1
[junit4:junit4] 2> 64678 T318 oasc.ZkController.register We are http://127.0.0.1:37678/mj/e/collection1/ and leader is http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 64678 T318 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37678/mj/e
[junit4:junit4] 2> 64678 T318 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 64678 T318 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C74 name=collection1 org.apache.solr.core.SolrCore@2d8c27d0 url=http://127.0.0.1:37678/mj/e/collection1 node=127.0.0.1:37678_mj%2Fe C74_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:37678_mj%2Fe, base_url=http://127.0.0.1:37678/mj/e}
[junit4:junit4] 2> 64679 T320 C74 P37678 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 64679 T320 C74 P37678 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 64679 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 64679 T320 C74 P37678 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 64680 T320 C74 P37678 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 64681 T259 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 64681 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 64681 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 64682 T320 C74 P37678 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 64687 T294 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:37678_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 64751 T259 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 64754 T259 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34628
[junit4:junit4] 2> 64755 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 64755 T259 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 64755 T259 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894
[junit4:junit4] 2> 64756 T259 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/solr.xml
[junit4:junit4] 2> 64756 T259 oasc.CoreContainer.<init> New CoreContainer 666034467
[junit4:junit4] 2> 64756 T259 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/'
[junit4:junit4] 2> 64757 T259 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/'
[junit4:junit4] 2> 64805 T259 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 64806 T259 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 64806 T259 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 64807 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 64807 T259 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 64808 T259 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 64808 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 64808 T259 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 64809 T259 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 64809 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 64812 T259 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 64813 T259 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54516/solr
[junit4:junit4] 2> 64813 T259 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 64814 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 64816 T332 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74dab801 name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 64817 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 64819 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 64825 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 64829 T334 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fe41459 name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 64829 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 64835 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 65013 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 65014 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:37678_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37678_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37678/mj/e"}
[junit4:junit4] 2> 65019 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65019 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65019 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65019 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65019 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65688 T294 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:37678_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 65688 T294 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:37678_mj%252Fe_collection1&state=recovering&nodeName=127.0.0.1:37678_mj%252Fe&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 65839 T259 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34628_mj%2Fe
[junit4:junit4] 2> 65841 T259 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34628_mj%2Fe
[junit4:junit4] 2> 65843 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65844 T301 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 65844 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 65844 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 65845 T317 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 65845 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 65846 T281 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 65846 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 65846 T288 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 65848 T334 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 65853 T335 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 65853 T335 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 66527 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 66528 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34628_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34628/mj/e"}
[junit4:junit4] 2> 66528 T282 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 66528 T282 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 66535 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 66535 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 66535 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 66535 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 66535 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 66857 T335 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/collection1
[junit4:junit4] 2> 66858 T335 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 66859 T335 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 66859 T335 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 66861 T335 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/collection1/'
[junit4:junit4] 2> 66861 T335 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/collection1/lib/README' to classloader
[junit4:junit4] 2> 66862 T335 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 66898 T335 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 66937 T335 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 66940 T335 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 66947 T335 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 67267 T335 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 67273 T335 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 67283 T335 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 67305 T335 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 67309 T335 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 67312 T335 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 67314 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 67315 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 67316 T335 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 67317 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 67318 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 67318 T335 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 67318 T335 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372068101894/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/
[junit4:junit4] 2> 67319 T335 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e7bc1f9
[junit4:junit4] 2> 67319 T335 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 67319 T335 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3
[junit4:junit4] 2> 67320 T335 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index/
[junit4:junit4] 2> 67320 T335 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 67320 T335 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index
[junit4:junit4] 2> 67324 T335 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 67324 T335 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 67326 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 67326 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 67327 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 67328 T335 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 67328 T335 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 67328 T335 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 67329 T335 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 67329 T335 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 67330 T335 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 67332 T335 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 67335 T335 oass.SolrIndexSearcher.<init> Opening Searcher@2e3cf0f6 main
[junit4:junit4] 2> 67336 T335 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 67336 T335 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 67342 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e3cf0f6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 67346 T335 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 67346 T335 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34628/mj/e collection:collection1 shard:shard1
[junit4:junit4] 2> 67350 T335 oasc.ZkController.register We are http://127.0.0.1:34628/mj/e/collection1/ and leader is http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 67350 T335 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34628/mj/e
[junit4:junit4] 2> 67350 T335 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 67351 T335 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C75 name=collection1 org.apache.solr.core.SolrCore@ae2d801 url=http://127.0.0.1:34628/mj/e/collection1 node=127.0.0.1:34628_mj%2Fe C75_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34628_mj%2Fe, base_url=http://127.0.0.1:34628/mj/e}
[junit4:junit4] 2> 67351 T337 C75 P34628 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 67352 T337 C75 P34628 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 67352 T335 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 67352 T337 C75 P34628 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 67353 T337 C75 P34628 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 67353 T259 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 67354 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 67354 T337 C75 P34628 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 67354 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 67360 T294 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:34628_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 67429 T259 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 67432 T259 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39259
[junit4:junit4] 2> 67433 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 67433 T259 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 67434 T259 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567
[junit4:junit4] 2> 67434 T259 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/solr.xml
[junit4:junit4] 2> 67435 T259 oasc.CoreContainer.<init> New CoreContainer 2013019297
[junit4:junit4] 2> 67435 T259 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/'
[junit4:junit4] 2> 67436 T259 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/'
[junit4:junit4] 2> 67488 T259 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 67489 T259 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 67489 T259 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 67489 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 67490 T259 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 67490 T259 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 67490 T259 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 67491 T259 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 67491 T259 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 67491 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 67495 T259 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 67495 T259 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54516/solr
[junit4:junit4] 2> 67496 T259 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 67497 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 67498 T349 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9c2595f name:ZooKeeperConnection Watcher:127.0.0.1:54516 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 67499 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 67505 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 67509 T259 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 67511 T351 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6958929a name:ZooKeeperConnection Watcher:127.0.0.1:54516/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 67511 T259 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 67516 T259 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C76 name=collection1 org.apache.solr.core.SolrCore@2d8c27d0 url=http://127.0.0.1:37678/mj/e/collection1 node=127.0.0.1:37678_mj%2Fe C76_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:37678_mj%2Fe, base_url=http://127.0.0.1:37678/mj/e}
[junit4:junit4] 2> 67689 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60141/mj/e/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 67689 T320 C76 P37678 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37678/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 67690 T320 C76 P37678 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 67690 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 67690 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 67691 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 67691 T320 C76 P37678 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 67691 T320 C76 P37678 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60141/mj/e/collection1/. core=collection1
[junit4:junit4] 2> 67691 T320 C76 P37678 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C77 name=collection1 org.apache.solr.core.SolrCore@13abd096 url=http://127.0.0.1:60141/mj/e/collection1 node=127.0.0.1:60141_mj%2Fe C77_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60141_mj%2Fe, base_url=http://127.0.0.1:60141/mj/e, leader=true}
[junit4:junit4] 2> 67697 T296 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 67698 T295 C77 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 67701 T295 C77 P60141 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 67701 T295 C77 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 67711 T295 C77 P60141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 67711 T295 C77 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 67712 T295 C77 P60141 oass.SolrIndexSearcher.<init> Opening Searcher@6e5f5971 realtime
[junit4:junit4] 2> 67712 T295 C77 P60141 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 67713 T295 C77 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 67714 T320 C76 P37678 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 67715 T320 C76 P37678 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 67717 T296 C77 P60141 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 67718 T296 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 67718 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 67719 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 67719 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 67721 T295 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 67722 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 67723 T320 C76 P37678 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931
[junit4:junit4] 2> 67723 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931 lockFactory=org.apache.lucene.store.NativeFSLockFactory@37dfac83; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 67726 T297 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 67728 T320 C76 P37678 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 67731 T320 C76 P37678 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 67732 T320 C76 P37678 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 67734 T320 C76 P37678 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 67734 T320 C76 P37678 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 67735 T320 C76 P37678 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 67735 T320 C76 P37678 oass.SolrIndexSearcher.<init> Opening Searcher@6a1e7a3e main
[junit4:junit4] 2> 67736 T319 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a1e7a3e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 67737 T320 C76 P37678 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931;done=true>>]
[junit4:junit4] 2> 67737 T320 C76 P37678 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931
[junit4:junit4] 2> 67737 T320 C76 P37678 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index.20130624130144931
[junit4:junit4] 2> 67740 T320 C76 P37678 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 67740 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 67741 T320 C76 P37678 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 67741 T320 C76 P37678 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 67743 T320 C76 P37678 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 68041 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 68043 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34628_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34628_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34628/mj/e"}
[junit4:junit4] 2> 68047 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:37678_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37678_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37678/mj/e"}
[junit4:junit4] 2> 68071 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68071 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68071 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68071 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68071 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68071 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68361 T294 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:34628_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 68361 T294 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:34628_mj%252Fe_collection1&state=recovering&nodeName=127.0.0.1:34628_mj%252Fe&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 68519 T259 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39259_mj%2Fe
[junit4:junit4] 2> 68521 T259 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39259_mj%2Fe
[junit4:junit4] 2> 68524 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68524 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68524 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 68525 T301 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68525 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 68526 T281 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68525 T317 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68526 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 68527 T351 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68526 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 68528 T334 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68528 T288 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 68535 T354 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 68536 T354 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 69580 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 69581 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39259_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39259/mj/e"}
[junit4:junit4] 2> 69581 T282 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 69581 T282 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 69586 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 69586 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 69587 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 69586 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 69586 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 69586 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> ASYNC NEW_CORE C78 name=collection1 org.apache.solr.core.SolrCore@ae2d801 url=http://127.0.0.1:34628/mj/e/collection1 node=127.0.0.1:34628_mj%2Fe C78_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34628_mj%2Fe, base_url=http://127.0.0.1:34628/mj/e}
[junit4:junit4] 2> 70362 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60141/mj/e/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 70363 T337 C78 P34628 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34628/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 70363 T337 C78 P34628 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 70364 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 70364 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 70364 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 70365 T337 C78 P34628 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 70365 T337 C78 P34628 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60141/mj/e/collection1/. core=collection1
[junit4:junit4] 2> 70365 T337 C78 P34628 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 70366 T294 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 70370 T296 C77 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 70381 T296 C77 P60141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 70382 T296 C77 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 70382 T296 C77 P60141 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 70383 T296 C77 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 70384 T337 C78 P34628 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 70384 T337 C78 P34628 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 70387 T297 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 70388 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 70388 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 70388 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 70391 T294 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 70391 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 70392 T337 C78 P34628 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601
[junit4:junit4] 2> 70393 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdb97c8; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 70396 T296 C77 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 70397 T337 C78 P34628 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 70399 T337 C78 P34628 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 70400 T337 C78 P34628 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 70403 T337 C78 P34628 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 70403 T337 C78 P34628 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 70403 T337 C78 P34628 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 70404 T337 C78 P34628 oass.SolrIndexSearcher.<init> Opening Searcher@7230424d main
[junit4:junit4] 2> 70405 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7230424d main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 70406 T337 C78 P34628 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601;done=true>>]
[junit4:junit4] 2> 70406 T337 C78 P34628 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601
[junit4:junit4] 2> 70407 T337 C78 P34628 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index.20130624130147601
[junit4:junit4] 2> 70407 T337 C78 P34628 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 70407 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 70407 T337 C78 P34628 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 70408 T337 C78 P34628 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 70409 T337 C78 P34628 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 70538 T354 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/collection1
[junit4:junit4] 2> 70538 T354 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 70539 T354 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 70539 T354 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 70541 T354 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/collection1/'
[junit4:junit4] 2> 70541 T354 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/collection1/lib/README' to classloader
[junit4:junit4] 2> 70542 T354 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 70583 T354 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 70617 T354 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 70619 T354 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 70628 T354 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 70976 T354 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 70983 T354 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 70985 T354 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 71005 T354 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 71009 T354 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 71013 T354 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 71014 T354 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 71015 T354 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 71015 T354 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 71017 T354 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 71017 T354 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 71017 T354 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 71018 T354 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372068104567/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/
[junit4:junit4] 2> 71018 T354 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e7bc1f9
[junit4:junit4] 2> 71019 T354 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 71019 T354 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4
[junit4:junit4] 2> 71020 T354 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index/
[junit4:junit4] 2> 71020 T354 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 71021 T354 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index
[junit4:junit4] 2> 71042 T354 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 71043 T354 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 71045 T354 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 71046 T354 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 71046 T354 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 71047 T354 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 71048 T354 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 71048 T354 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 71050 T354 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 71050 T354 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 71051 T354 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 71054 T354 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 71059 T354 oass.SolrIndexSearcher.<init> Opening Searcher@2198ffb6 main
[junit4:junit4] 2> 71060 T354 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 71060 T354 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 71068 T357 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2198ffb6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 71072 T354 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 71072 T354 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39259/mj/e collection:collection1 shard:shard1
[junit4:junit4] 2> 71076 T354 oasc.ZkController.register We are http://127.0.0.1:39259/mj/e/collection1/ and leader is http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 71077 T354 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39259/mj/e
[junit4:junit4] 2> 71077 T354 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 71077 T354 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C79 name=collection1 org.apache.solr.core.SolrCore@3139be78 url=http://127.0.0.1:39259/mj/e/collection1 node=127.0.0.1:39259_mj%2Fe C79_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:39259_mj%2Fe, base_url=http://127.0.0.1:39259/mj/e}
[junit4:junit4] 2> 71078 T358 C79 P39259 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 71079 T358 C79 P39259 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 71079 T354 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 71079 T358 C79 P39259 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 71080 T358 C79 P39259 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 71080 T259 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 71081 T259 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 71081 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 71086 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 71086 T358 C79 P39259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 71090 T259 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 71091 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 71091 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 71092 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 71093 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34628_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34628_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34628/mj/e"}
[junit4:junit4] 2> 71093 T296 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:39259_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 71097 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:39259_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39259_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39259/mj/e"}
[junit4:junit4] 2> 71102 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 71102 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 71102 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 71102 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 71102 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 71103 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 72094 T296 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:39259_mj%2Fe_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 72094 T296 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:39259_mj%252Fe_collection1&state=recovering&nodeName=127.0.0.1:39259_mj%252Fe&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 72094 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 73097 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C79_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:39259_mj%2Fe, base_url=http://127.0.0.1:39259/mj/e}
[junit4:junit4] 2> 74095 T358 C79 P39259 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60141/mj/e/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 74096 T358 C79 P39259 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39259/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 74096 T358 C79 P39259 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 74096 T358 C79 P39259 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 74097 T358 C79 P39259 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 74097 T358 C79 P39259 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 74097 T358 C79 P39259 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 74097 T358 C79 P39259 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60141/mj/e/collection1/. core=collection1
[junit4:junit4] 2> 74097 T358 C79 P39259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 74098 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C80 name=collection1 org.apache.solr.core.SolrCore@13abd096 url=http://127.0.0.1:60141/mj/e/collection1 node=127.0.0.1:60141_mj%2Fe C80_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60141_mj%2Fe, base_url=http://127.0.0.1:60141/mj/e, leader=true}
[junit4:junit4] 2> 74099 T296 C80 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 74102 T295 C80 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 74111 T295 C80 P60141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 74112 T295 C80 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 74112 T295 C80 P60141 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 74113 T295 C80 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 74114 T358 C79 P39259 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 74114 T358 C79 P39259 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 74116 T294 C80 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 74117 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 74117 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 74117 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 74120 T296 C80 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 74121 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 74122 T358 C79 P39259 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330
[junit4:junit4] 2> 74122 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5a7dc544; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 74125 T295 C80 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1
[junit4:junit4] 2> 74126 T358 C79 P39259 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 74130 T358 C79 P39259 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 74130 T358 C79 P39259 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 74132 T358 C79 P39259 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> ASYNC NEW_CORE C81 name=collection1 org.apache.solr.core.SolrCore@3139be78 url=http://127.0.0.1:39259/mj/e/collection1 node=127.0.0.1:39259_mj%2Fe C81_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:39259_mj%2Fe, base_url=http://127.0.0.1:39259/mj/e}
[junit4:junit4] 2> 74140 T358 C81 P39259 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 74140 T358 C81 P39259 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 74141 T358 C81 P39259 oass.SolrIndexSearcher.<init> Opening Searcher@469a35c3 main
[junit4:junit4] 2> 74142 T357 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@469a35c3 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 74143 T358 C81 P39259 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330;done=true>>]
[junit4:junit4] 2> 74143 T358 C81 P39259 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330
[junit4:junit4] 2> 74144 T358 C81 P39259 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index.20130624130151330
[junit4:junit4] 2> 74144 T358 C81 P39259 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 74144 T358 C81 P39259 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 74144 T358 C81 P39259 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 74145 T358 C81 P39259 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 74146 T358 C81 P39259 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 75101 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 75615 T282 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 75616 T282 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:39259_mj%2Fe_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39259_mj%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39259/mj/e"}
[junit4:junit4] 2> 75620 T288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 75620 T317 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 75621 T334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 75620 T281 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 75620 T301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 75620 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 76104 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 76105 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C82 name=collection1 org.apache.solr.core.SolrCore@7b76dced url=http://127.0.0.1:45893/mj/e/collection1 node=127.0.0.1:45893_mj%2Fe C82_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:45893_mj%2Fe, base_url=http://127.0.0.1:45893/mj/e, leader=true}
[junit4:junit4] 2> 76112 T276 C82 P45893 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 76116 T276 C82 P45893 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 76117 T276 C82 P45893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 76131 T276 C82 P45893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 76132 T276 C82 P45893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 76133 T276 C82 P45893 oass.SolrIndexSearcher.<init> Opening Searcher@7cd62439 main
[junit4:junit4] 2> 76133 T276 C82 P45893 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 76134 T285 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7cd62439 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 76135 T276 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> C81_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39259_mj%2Fe, base_url=http://127.0.0.1:39259/mj/e}
[junit4:junit4] 2> 76145 T347 C81 P39259 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60141/mj/e/collection1/, StdNode: http://127.0.0.1:37678/mj/e/collection1/, StdNode: http://127.0.0.1:34628/mj/e/collection1/, StdNode: http://127.0.0.1:39259/mj/e/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C83 name=collection1 org.apache.solr.core.SolrCore@13abd096 url=http://127.0.0.1:60141/mj/e/collection1 node=127.0.0.1:60141_mj%2Fe C83_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60141_mj%2Fe, base_url=http://127.0.0.1:60141/mj/e, leader=true}
[junit4:junit4] 2> 76150 T297 C83 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 76152 T346 C81 P39259 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C84 name=collection1 org.apache.solr.core.SolrCore@ae2d801 url=http://127.0.0.1:34628/mj/e/collection1 node=127.0.0.1:34628_mj%2Fe C84_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34628_mj%2Fe, base_url=http://127.0.0.1:34628/mj/e}
[junit4:junit4] 2> 76157 T327 C84 P34628 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C85 name=collection1 org.apache.solr.core.SolrCore@2d8c27d0 url=http://127.0.0.1:37678/mj/e/collection1 node=127.0.0.1:37678_mj%2Fe C85_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37678_mj%2Fe, base_url=http://127.0.0.1:37678/mj/e}
[junit4:junit4] 2> 76159 T311 C85 P37678 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 76171 T297 C83 P60141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 76172 T297 C83 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 76172 T297 C83 P60141 oass.SolrIndexSearcher.<init> Opening Searcher@53d653ef main
[junit4:junit4] 2> 76173 T297 C83 P60141 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 76173 T303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53d653ef main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 76174 T297 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 76201 T346 C81 P39259 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 76202 T346 C81 P39259 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 76201 T311 C85 P37678 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 76201 T327 C84 P34628 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 76203 T327 C84 P34628 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 76202 T346 C81 P39259 oass.SolrIndexSearcher.<init> Opening Searcher@7043c62c main
[junit4:junit4] 2> 76202 T311 C85 P37678 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 76204 T346 C81 P39259 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 76204 T311 C85 P37678 oass.SolrIndexSearcher.<init> Opening Searcher@3893055d main
[junit4:junit4] 2> 76204 T327 C84 P34628 oass.SolrIndexSearcher.<init> Opening Searcher@4f4fb284 main
[junit4:junit4] 2> 76205 T311 C85 P37678 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 76205 T357 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7043c62c main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 76206 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f4fb284 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 76205 T319 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3893055d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 76205 T327 C84 P34628 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 76207 T346 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> 76207 T327 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 76207 T311 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 76208 T347 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 76210 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 76212 T296 C83 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 76215 T310 C85 P37678 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 76217 T328 C84 P34628 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 76219 T345 C81 P39259 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 78224 T275 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438717696207224832)} 0 1
[junit4:junit4] 2> 78235 T345 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={update.distrib=FROMLEADER&_version_=-1438717696213516288&update.from=http://127.0.0.1:60141/mj/e/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438717696213516288)} 0 1
[junit4:junit4] 2> 78235 T329 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={update.distrib=FROMLEADER&_version_=-1438717696213516288&update.from=http://127.0.0.1:60141/mj/e/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438717696213516288)} 0 1
[junit4:junit4] 2> 78237 T312 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={update.distrib=FROMLEADER&_version_=-1438717696213516288&update.from=http://127.0.0.1:60141/mj/e/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438717696213516288)} 0 1
[junit4:junit4] 2> 78238 T296 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438717696213516288)} 0 9
[junit4:junit4] 2> 78239 T313 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 12
[junit4:junit4] 2> 78246 T276 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[0 (1438717696227147776)]} 0 4
[junit4:junit4] 2> 78261 T327 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438717696237633536)]} 0 4
[junit4:junit4] 2> 78262 T347 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438717696237633536)]} 0 4
[junit4:junit4] 2> 78263 T311 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438717696237633536)]} 0 5
[junit4:junit4] 2> 78264 T295 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:39259/mj/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438717696237633536)]} 0 12
[junit4:junit4] 2> 78264 T346 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[0]} 0 15
[junit4:junit4] 2> 78270 T277 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[1 (1438717696253362176)]} 0 3
[junit4:junit4] 2> 78284 T330 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438717696262799360)]} 0 3
[junit4:junit4] 2> 78285 T310 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438717696262799360)]} 0 4
[junit4:junit4] 2> 78285 T345 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438717696262799360)]} 0 4
[junit4:junit4] 2> 78286 T297 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:39259/mj/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438717696262799360)]} 0 10
[junit4:junit4] 2> 78286 T344 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[1]} 0 12
[junit4:junit4] 2> 78292 T273 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[2 (1438717696277479424)]} 0 3
[junit4:junit4] 2> 78297 T296 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:37678/mj/e/collection1/]
[junit4:junit4] 2> 78297 T296 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:37678/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:true
[junit4:junit4] 2> 78297 T296 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:34628/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:false
[junit4:junit4] 2> 78297 T296 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39259/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:false
[junit4:junit4] 2> 78304 T347 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[2 (1438717696285868032)]} 0 2
[junit4:junit4] 2> 78304 T327 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[2 (1438717696285868032)]} 0 2
[junit4:junit4] 2> 78305 T296 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:34628/mj/e/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[2 (1438717696285868032)]} 0 9
[junit4:junit4] 2> 78306 T329 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[2]} 0 12
[junit4:junit4] 2> 78310 T275 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={wt=javabin&version=2} {add=[3 (1438717696296353792)]} 0 2
[junit4:junit4] 2> 78316 T295 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:37678/mj/e/collection1/]
[junit4:junit4] 2> 78316 T295 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:37678/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:true
[junit4:junit4] 2> 78316 T295 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:34628/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:false
[junit4:junit4] 2> 78317 T295 C83 P60141 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39259/mj/e/collection1/ against:[http://127.0.0.1:37678/mj/e/collection1/] result:false
[junit4:junit4] 2> 78323 T346 C81 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[3 (1438717696305790976)]} 0 2
[junit4:junit4] 2> 78324 T330 C84 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:60141/mj/e/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[3 (1438717696305790976)]} 0 2
[junit4:junit4] 2> 78325 T295 C83 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={distrib.from=http://127.0.0.1:37678/mj/e/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&wt=javabin&version=2} {add=[3 (1438717696305790976)]} 0 10
[junit4:junit4] 2> 78326 T312 C85 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={test.distrib.skip.servers=http://127.0.0.1:37678/mj/e/collection1/&test.distrib.skip.servers=http://127.0.0.1:34628/mj/e/collection1/&wt=javabin&version=2} {add=[3]} 0 13
[junit4:junit4] 2> 78329 T276 C82 P45893 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78361 T276 C82 P45893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 78362 T276 C82 P45893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 78384 T276 C82 P45893 oass.SolrIndexSearcher.<init> Opening Searcher@528fad8e main
[junit4:junit4] 2> 78385 T276 C82 P45893 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78385 T285 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@528fad8e main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 78386 T276 C82 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 57
[junit4:junit4] 2> 78389 T345 C81 P39259 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60141/mj/e/collection1/, StdNode: http://127.0.0.1:37678/mj/e/collection1/, StdNode: http://127.0.0.1:34628/mj/e/collection1/, StdNode: http://127.0.0.1:39259/mj/e/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 78392 T344 C81 P39259 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78392 T328 C84 P34628 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78392 T297 C83 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78393 T313 C85 P37678 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C86 name=collection1 org.apache.solr.core.SolrCore@3139be78 url=http://127.0.0.1:39259/mj/e/collection1 node=127.0.0.1:39259_mj%2Fe C86_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39259_mj%2Fe, base_url=http://127.0.0.1:39259/mj/e}
[junit4:junit4] 2> 78434 T344 C86 P39259 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 78435 T344 C86 P39259 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 78440 T344 C86 P39259 oass.SolrIndexSearcher.<init> Opening Searcher@6fba6e81 main
[junit4:junit4] 2> 78441 T344 C86 P39259 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C87 name=collection1 org.apache.solr.core.SolrCore@2d8c27d0 url=http://127.0.0.1:37678/mj/e/collection1 node=127.0.0.1:37678_mj%2Fe C87_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37678_mj%2Fe, base_url=http://127.0.0.1:37678/mj/e}
[junit4:junit4] 2> 78441 T313 C87 P37678 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@705e4f14; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 78442 T313 C87 P37678 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 78442 T357 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6fba6e81 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 78443 T344 C86 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 78446 T313 C87 P37678 oass.SolrIndexSearcher.<init> Opening Searcher@2db4e510 main
[junit4:junit4] 2> 78447 T313 C87 P37678 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78448 T319 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2db4e510 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):C2)}
[junit4:junit4] 2> 78449 T313 C87 P37678 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 56
[junit4:junit4] 2> ASYNC NEW_CORE C88 name=collection1 org.apache.solr.core.SolrCore@ae2d801 url=http://127.0.0.1:34628/mj/e/collection1 node=127.0.0.1:34628_mj%2Fe C88_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34628_mj%2Fe, base_url=http://127.0.0.1:34628/mj/e}
[junit4:junit4] 2> 78461 T328 C88 P34628 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 78461 T328 C88 P34628 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 78465 T328 C88 P34628 oass.SolrIndexSearcher.<init> Opening Searcher@269ad66e main
[junit4:junit4] 2> 78466 T328 C88 P34628 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78466 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@269ad66e main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 78467 T328 C88 P34628 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 75
[junit4:junit4] 2> ASYNC NEW_CORE C89 name=collection1 org.apache.solr.core.SolrCore@13abd096 url=http://127.0.0.1:60141/mj/e/collection1 node=127.0.0.1:60141_mj%2Fe C89_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60141_mj%2Fe, base_url=http://127.0.0.1:60141/mj/e, leader=true}
[junit4:junit4] 2> 78471 T297 C89 P60141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2101f0d8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 78472 T297 C89 P60141 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 78475 T297 C89 P60141 oass.SolrIndexSearcher.<init> Opening Searcher@358c5162 main
[junit4:junit4] 2> 78476 T297 C89 P60141 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78477 T303 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@358c5162 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 78477 T297 C89 P60141 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 85
[junit4:junit4] 2> 78478 T345 C86 P39259 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 89
[junit4:junit4] 2> 78479 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 78480 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 78481 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 78484 T294 C89 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 78487 T311 C87 P37678 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=1
[junit4:junit4] 2> 78490 T327 C88 P34628 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 78492 T347 C86 P39259 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 78493 T259 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 78498 T329 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 78498 T329 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 78505 T295 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 78505 T295 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 78505 T295 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e START replicas=[http://127.0.0.1:37678/mj/e/collection1/, http://127.0.0.1:34628/mj/e/collection1/, http://127.0.0.1:39259/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 78508 T310 C87 P37678 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78508 T328 C88 P34628 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78509 T344 C86 P39259 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 78509 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Received 3 versions from 127.0.0.1:37678/mj/e/collection1/
[junit4:junit4] 2> 78509 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Our versions are newer. ourLowThreshold=1438717696213516288 otherHigh=1438717696262799360
[junit4:junit4] 2> 78510 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Received 5 versions from 127.0.0.1:34628/mj/e/collection1/
[junit4:junit4] 2> 78510 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Our versions are newer. ourLowThreshold=1438717696213516288 otherHigh=1438717696285868032
[junit4:junit4] 2> 78510 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Received 5 versions from 127.0.0.1:39259/mj/e/collection1/
[junit4:junit4] 2> 78510 T295 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e Our versions are newer. ourLowThreshold=1438717696213516288 otherHigh=1438717696285868032
[junit4:junit4] 2> 78511 T295 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60141/mj/e DONE. sync succeeded
[junit4:junit4] 2> 78511 T295 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 78511 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: try and ask http://127.0.0.1:37678/mj/e/collection1/ to sync
[junit4:junit4] 2> 78512 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: try and ask http://127.0.0.1:34628/mj/e/collection1/ to sync
[junit4:junit4] 2> 78512 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: try and ask http://127.0.0.1:39259/mj/e/collection1/ to sync
[junit4:junit4] 2> 78514 T313 C87 P37678 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37678/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 78514 T327 C88 P34628 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34628/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 78515 T347 C86 P39259 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39259/mj/e START replicas=[http://127.0.0.1:60141/mj/e/collection1/] nUpdates=100
[junit4:junit4] 2> 78517 T297 C89 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78518 T294 C89 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 78517 T296 C89 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78518 T327 C88 P34628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34628/mj/e Received 5 versions from 127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 78519 T313 C87 P37678 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37678/mj/e Received 5 versions from 127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 78519 T327 C88 P34628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34628/mj/e Our versions are newer. ourLowThreshold=1438717696213516288 otherHigh=1438717696285868032
[junit4:junit4] 2> 78520 T327 C88 P34628 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34628/mj/e DONE. sync succeeded
[junit4:junit4] 2> 78519 T347 C86 P39259 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39259/mj/e Received 5 versions from 127.0.0.1:60141/mj/e/collection1/
[junit4:junit4] 2> 78520 T327 C88 P34628 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={sync=http://127.0.0.1:60141/mj/e/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=6
[junit4:junit4] 2> 78520 T313 C87 P37678 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:37678/mj/e Requesting updates from 127.0.0.1:60141/mj/e/collection1/n=2 versions=[1438717696305790976, 1438717696285868032]
[junit4:junit4] 2> 78521 T347 C86 P39259 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39259/mj/e Our versions are newer. ourLowThreshold=1438717696213516288 otherHigh=1438717696285868032
[junit4:junit4] 2> 78522 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: sync completed with http://127.0.0.1:34628/mj/e/collection1/
[junit4:junit4] 2> 78522 T347 C86 P39259 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39259/mj/e DONE. sync succeeded
[junit4:junit4] 2> 78522 T347 C86 P39259 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={sync=http://127.0.0.1:60141/mj/e/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7
[junit4:junit4] 2> 78523 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: sync completed with http://127.0.0.1:39259/mj/e/collection1/
[junit4:junit4] 2> 78524 T294 C89 P60141 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={distrib=false&getUpdates=1438717696305790976,1438717696285868032&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78530 T313 C87 P37678 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438717696285868032), 3 (1438717696305790976)]} 0 5
[junit4:junit4] 2> 78531 T313 C87 P37678 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37678/mj/e DONE. sync succeeded
[junit4:junit4] 2> 78531 T313 C87 P37678 oasc.SolrCore.execute [collection1] webapp=/mj/e path=/get params={sync=http://127.0.0.1:60141/mj/e/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=17
[junit4:junit4] 2> 78532 T295 oasc.SyncStrategy.syncToMe http://127.0.0.1:60141/mj/e/collection1/: sync completed with http://127.0.0.1:37678/mj/e/collection1/
[junit4:junit4] 2> 78532 T295 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=27
[junit4:junit4] 2> 78533 T329 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=35
[junit4:junit4] 2> 78533 T259 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 78534 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 78535 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 78536 T259 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C90 name=collection1 org.apache.solr.core.SolrCore@7b76dced url=http://127.0.0.1:45893/mj/e/collection1 node=127.0.0.1:45893_mj%2Fe C90_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:45893_mj%2Fe, base_url=http://127.0.0.1:45893/mj/e, leader=true}
[junit4:junit4] 2> 78538 T277 C90 P45893 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78581 T277 C90 P45893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4980c2b4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 78582 T277 C90 P45893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 78582 T277 C90 P45893 oass.SolrIndexSearcher.<init> Opening Searcher@447532fb main
[junit4:junit4] 2> 78583 T277 C90 P45893 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78584 T285 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@447532fb main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 78584 T277 C90 P45893 oasup.LogUpdateProcessor.finish [collection1] webapp=/mj/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 46
[junit4:junit4] 2> 78587 T346 C86 P39259 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60141/mj/e/collection1/, StdNode: http://127.0.0.1:37678/mj/e/collection1/, StdNode: http://127.0.0.1:34628/mj/e/collection1/, StdNode: http://127.0.0.1:39259/mj/e/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 78590 T311 C87 P37678 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78590 T344 C86 P39259 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78590 T297 C89 P60141 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78590 T330 C88 P34628 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78621 T330 C88 P34628 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@655ecf0e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 78622 T330 C88 P34628 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 78621 T344 C86 P39259 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372068092747/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396d332d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7,generation=7}
[junit4:junit4] 2> 78622 T344 C86 P39259 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 78622 T330 C88 P34628 oass.SolrIndexSearcher.<init> Opening Searcher@7a93d960 main
[junit4:junit4] 2> 78623 T344 C86 P39259 oass.SolrIndexSearcher.<init> Opening Searcher@2acf062d main
[junit4:junit4] 2> 78623 T330 C88 P34628 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78623 T344 C86 P39259 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78624 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a93d960 main{StandardDirectoryReader(segme
[...truncated too long message...]
"state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "127.0.0.1:60141_mj%2Fe_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:60141_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60141/mj/e"},
[junit4:junit4] 1> "127.0.0.1:37678_mj%2Fe_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:37678_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37678/mj/e"},
[junit4:junit4] 1> "127.0.0.1:34628_mj%2Fe_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34628_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34628/mj/e"},
[junit4:junit4] 1> "127.0.0.1:39259_mj%2Fe_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:39259_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39259/mj/e",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:45893_mj%2Fe_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:45893_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:45893/mj/e",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:39259_mj%2Fe (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:34628_mj%2Fe (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:45893_mj%2Fe (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:60141_mj%2Fe (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919854526595084-127.0.0.1:39259_mj%2Fe_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919854526595086-127.0.0.1:60141_mj%2Fe_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919854526595082-127.0.0.1:34628_mj%2Fe_collection1-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:39259_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39259/mj/e"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89919854526595075-127.0.0.1:45893_mj%2Fe_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:45893_mj%2Fe",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:45893/mj/e"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89919854526595086-127.0.0.1:60141_mj%2Fe-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919854526595084-127.0.0.1:39259_mj%2Fe-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919854526595075-127.0.0.1:45893_mj%2Fe-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919854526595082-127.0.0.1:34628_mj%2Fe-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89919854526595075-127.0.0.1:45893_mj%2Fe-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=15851157ED6233A5 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sr_BA -Dtests.timezone=Europe/Chisinau -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 102s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:60141/mj/e/collection1lastClient and got 252 from http://127.0.0.1:34628/mj/e/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([15851157ED6233A5:94639F4F9A3D5399]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 157958 T259 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 102431 T258 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=sr_BA, timezone=Europe/Chisinau
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=8,threads=1,free=55883928,total=126746624
[junit4:junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaTest, AlternateDirectoryTest, TestSolrIndexConfig, ChaosMonkeyNothingIsSafeTest, AliasIntegrationTest, MBeansHandlerTest, CircularListTest, TestSurroundQueryParser, TestFunctionQuery, DistributedQueryElevationComponentTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 103.46s, 1 test, 1 failure <<< FAILURES!
[...truncated 807 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:366: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1243: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 35 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure