You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/18 22:25:19 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 574 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/574/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<179> but was:<33>
Stack Trace:
java.lang.AssertionError: expected:<179> but was:<33>
at __randomizedtesting.SeedInfo.seed([D4A1FB632969ACEC:5547757B5E36CCD0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9611 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 2839216 T5735 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 2839225 T5735 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1371585502145
[junit4:junit4] 2> 2839227 T5735 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 2839228 T5736 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 2839329 T5735 oasc.ZkTestServer.run start zk server on port:51016
[junit4:junit4] 2> 2839335 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2839346 T5742 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fc63ad2 name:ZooKeeperConnection Watcher:127.0.0.1:51016 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2839346 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2839347 T5735 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 2839358 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2839365 T5744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20ce4c41 name:ZooKeeperConnection Watcher:127.0.0.1:51016/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2839365 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2839366 T5735 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 2839374 T5735 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 2839381 T5735 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 2839387 T5735 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 2839395 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2839396 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2839407 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 2839411 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 2839424 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2839425 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2839435 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 2839437 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 2839445 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 2839446 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 2839453 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2839454 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2839464 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2839465 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2839471 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2839472 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2839487 T5735 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2839492 T5735 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2840103 T5735 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2840130 T5735 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51019
[junit4:junit4] 2> 2840132 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2840133 T5735 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2840134 T5735 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421
[junit4:junit4] 2> 2840135 T5735 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/solr.xml
[junit4:junit4] 2> 2840136 T5735 oasc.CoreContainer.<init> New CoreContainer 485602329
[junit4:junit4] 2> 2840137 T5735 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/'
[junit4:junit4] 2> 2840138 T5735 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/'
[junit4:junit4] 2> 2840411 T5735 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2840412 T5735 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2840412 T5735 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2840413 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2840413 T5735 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2840414 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2840414 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2840415 T5735 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2840415 T5735 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2840416 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2840444 T5735 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2840445 T5735 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51016/solr
[junit4:junit4] 2> 2840445 T5735 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2840448 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2840459 T5755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69c859ad name:ZooKeeperConnection Watcher:127.0.0.1:51016 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2840461 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2840474 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2840501 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2840508 T5757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55484ca1 name:ZooKeeperConnection Watcher:127.0.0.1:51016/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2840508 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2840513 T5735 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 2840532 T5735 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 2840541 T5735 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 2840546 T5735 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51019_
[junit4:junit4] 2> 2840549 T5735 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51019_
[junit4:junit4] 2> 2840557 T5735 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 2840572 T5735 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 2840581 T5735 oasc.Overseer.start Overseer (id=89888227469426691-127.0.0.1:51019_-n_0000000000) starting
[junit4:junit4] 2> 2840599 T5735 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 2840619 T5759 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 2840626 T5735 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 2840643 T5735 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 2840647 T5735 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2840663 T5758 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 2840668 T5760 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2840669 T5760 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2842181 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2842186 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51019_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51019"}
[junit4:junit4] 2> 2842187 T5758 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2842189 T5758 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2842206 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2842673 T5760 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/collection1
[junit4:junit4] 2> 2842674 T5760 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 2842676 T5760 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2842676 T5760 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 2842681 T5760 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/collection1/'
[junit4:junit4] 2> 2842684 T5760 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2842685 T5760 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/collection1/lib/README' to classloader
[junit4:junit4] 2> 2842766 T5760 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2842863 T5760 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2842867 T5760 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2842880 T5760 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2844059 T5760 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2844076 T5760 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2844082 T5760 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2844127 T5760 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2844138 T5760 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2844147 T5760 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2844150 T5760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2844150 T5760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2844151 T5760 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2844154 T5760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2844154 T5760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2844155 T5760 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2844155 T5760 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371585502421/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data/
[junit4:junit4] 2> 2844155 T5760 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d5a5223
[junit4:junit4] 2> 2844156 T5760 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2844157 T5760 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data
[junit4:junit4] 2> 2844157 T5760 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data/index/
[junit4:junit4] 2> 2844158 T5760 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2844158 T5760 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data/index
[junit4:junit4] 2> 2844166 T5760 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@77b08ea0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@634c90a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2844166 T5760 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2844170 T5760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2844171 T5760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2844172 T5760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2844174 T5760 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2844175 T5760 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2844175 T5760 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2844176 T5760 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2844177 T5760 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2844178 T5760 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2844204 T5760 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2844216 T5760 oass.SolrIndexSearcher.<init> Opening Searcher@435a81d9 main
[junit4:junit4] 2> 2844216 T5760 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/control/data/tlog
[junit4:junit4] 2> 2844218 T5760 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2844219 T5760 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2844225 T5760 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2844226 T5760 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51019 collection:control_collection shard:shard1
[junit4:junit4] 2> 2844228 T5761 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@435a81d9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2844229 T5760 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2844248 T5760 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2844262 T5760 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2844263 T5760 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2844263 T5760 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51019/collection1/
[junit4:junit4] 2> 2844263 T5760 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2844264 T5760 oasc.SyncStrategy.syncToMe http://127.0.0.1:51019/collection1/ has no replicas
[junit4:junit4] 2> 2844264 T5760 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51019/collection1/
[junit4:junit4] 2> 2844264 T5760 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2845326 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2845351 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2845414 T5760 oasc.ZkController.register We are http://127.0.0.1:51019/collection1/ and leader is http://127.0.0.1:51019/collection1/
[junit4:junit4] 2> 2845416 T5760 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51019
[junit4:junit4] 2> 2845416 T5760 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2845416 T5760 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2845417 T5760 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2845421 T5760 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2845424 T5735 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2845424 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2845425 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2845459 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2845462 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2845468 T5764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@706d31b8 name:ZooKeeperConnection Watcher:127.0.0.1:51016/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2845468 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2845471 T5735 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2845477 T5735 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 2846056 T5735 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2846064 T5735 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51023
[junit4:junit4] 2> 2846066 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2846067 T5735 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2846070 T5735 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397
[junit4:junit4] 2> 2846071 T5735 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/solr.xml
[junit4:junit4] 2> 2846072 T5735 oasc.CoreContainer.<init> New CoreContainer 1344846032
[junit4:junit4] 2> 2846074 T5735 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/'
[junit4:junit4] 2> 2846075 T5735 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/'
[junit4:junit4] 2> 2846329 T5735 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2846330 T5735 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2846331 T5735 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2846331 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2846332 T5735 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2846332 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2846333 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2846333 T5735 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2846334 T5735 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2846335 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2846367 T5735 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2846368 T5735 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51016/solr
[junit4:junit4] 2> 2846369 T5735 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2846371 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2846376 T5775 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b39c693 name:ZooKeeperConnection Watcher:127.0.0.1:51016 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2846377 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2846381 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2846407 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2846412 T5777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b190e21 name:ZooKeeperConnection Watcher:127.0.0.1:51016/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2846413 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2846429 T5735 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2846866 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2846868 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51019__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51019_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51019"}
[junit4:junit4] 2> 2846886 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2846886 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2846886 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2847445 T5735 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51023_
[junit4:junit4] 2> 2847448 T5735 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51023_
[junit4:junit4] 2> 2847457 T5757 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2847457 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2847463 T5777 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2847464 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2847465 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2847470 T5764 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2847495 T5778 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2847496 T5778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2848400 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2848402 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51023_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51023"}
[junit4:junit4] 2> 2848402 T5758 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 2848402 T5758 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2848412 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2848415 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2848417 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2848500 T5778 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/collection1
[junit4:junit4] 2> 2848501 T5778 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2848504 T5778 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2848505 T5778 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2848508 T5778 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/collection1/'
[junit4:junit4] 2> 2848510 T5778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2848510 T5778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/collection1/lib/README' to classloader
[junit4:junit4] 2> 2848611 T5778 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2848724 T5778 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2848731 T5778 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2848746 T5778 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2850239 T5778 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2850264 T5778 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2850275 T5778 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2850327 T5778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2850335 T5778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2850344 T5778 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2850347 T5778 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2850349 T5778 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2850349 T5778 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2850353 T5778 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2850353 T5778 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2850353 T5778 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2850354 T5778 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371585508397/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1/
[junit4:junit4] 2> 2850354 T5778 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d5a5223
[junit4:junit4] 2> 2850355 T5778 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2850356 T5778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1
[junit4:junit4] 2> 2850357 T5778 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1/index/
[junit4:junit4] 2> 2850357 T5778 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2850358 T5778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1/index
[junit4:junit4] 2> 2850364 T5778 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c16266f lockFactory=org.apache.lucene.store.NativeFSLockFactory@43141ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2850364 T5778 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2850369 T5778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2850369 T5778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2850370 T5778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2850372 T5778 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2850373 T5778 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2850374 T5778 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2850374 T5778 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2850375 T5778 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2850376 T5778 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2850407 T5778 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2850418 T5778 oass.SolrIndexSearcher.<init> Opening Searcher@129b1552 main
[junit4:junit4] 2> 2850420 T5778 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty1/tlog
[junit4:junit4] 2> 2850421 T5778 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2850422 T5778 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2850428 T5779 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@129b1552 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2850433 T5778 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2850434 T5778 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51023 collection:collection1 shard:shard1
[junit4:junit4] 2> 2850437 T5778 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 2850459 T5778 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2850475 T5778 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2850476 T5778 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2850476 T5778 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51023/collection1/
[junit4:junit4] 2> 2850477 T5778 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2850477 T5778 oasc.SyncStrategy.syncToMe http://127.0.0.1:51023/collection1/ has no replicas
[junit4:junit4] 2> 2850478 T5778 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51023/collection1/
[junit4:junit4] 2> 2850478 T5778 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 2851468 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2851499 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2851504 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2851506 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2851545 T5778 oasc.ZkController.register We are http://127.0.0.1:51023/collection1/ and leader is http://127.0.0.1:51023/collection1/
[junit4:junit4] 2> 2851546 T5778 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51023
[junit4:junit4] 2> 2851546 T5778 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2851547 T5778 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2851547 T5778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2851552 T5778 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2851557 T5735 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2851559 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2851560 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2852256 T5735 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2852268 T5735 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51026
[junit4:junit4] 2> 2852269 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2852269 T5735 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2852270 T5735 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524
[junit4:junit4] 2> 2852271 T5735 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/solr.xml
[junit4:junit4] 2> 2852271 T5735 oasc.CoreContainer.<init> New CoreContainer 1803924789
[junit4:junit4] 2> 2852272 T5735 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/'
[junit4:junit4] 2> 2852272 T5735 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/'
[junit4:junit4] 2> 2852494 T5735 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2852496 T5735 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2852496 T5735 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2852497 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2852497 T5735 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2852498 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2852499 T5735 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2852499 T5735 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2852500 T5735 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2852501 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2852531 T5735 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2852532 T5735 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51016/solr
[junit4:junit4] 2> 2852533 T5735 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2852536 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2852543 T5791 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37033d5a name:ZooKeeperConnection Watcher:127.0.0.1:51016 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2852544 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2852554 T5737 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f58dddbce0007, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 2852555 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2852589 T5735 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2852594 T5793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b7e905a name:ZooKeeperConnection Watcher:127.0.0.1:51016/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2852595 T5735 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2852607 T5735 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2853035 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2853037 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51023__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51023_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51023"}
[junit4:junit4] 2> 2853050 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2853052 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2853055 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2853058 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2853621 T5735 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51026_
[junit4:junit4] 2> 2853624 T5735 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51026_
[junit4:junit4] 2> 2853629 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2853631 T5777 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2853632 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2853632 T5757 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2853632 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2853633 T5793 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2853633 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2853638 T5764 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2853650 T5794 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2853651 T5794 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2854572 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2854574 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51026_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51026"}
[junit4:junit4] 2> 2854575 T5758 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2854575 T5758 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2854595 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2854596 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2854596 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2854595 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2854656 T5794 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/collection1
[junit4:junit4] 2> 2854657 T5794 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2854659 T5794 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2854659 T5794 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2854662 T5794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/collection1/'
[junit4:junit4] 2> 2854664 T5794 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2854665 T5794 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/collection1/lib/README' to classloader
[junit4:junit4] 2> 2854759 T5794 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2854868 T5794 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2854871 T5794 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2854879 T5794 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2856011 T5794 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2856025 T5794 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2856040 T5794 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2856082 T5794 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2856092 T5794 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2856108 T5794 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2856112 T5794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2856113 T5794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2856264 T5794 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2856270 T5794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2856270 T5794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2856271 T5794 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2856271 T5794 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371585514524/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/
[junit4:junit4] 2> 2856271 T5794 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d5a5223
[junit4:junit4] 2> 2856272 T5794 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2856274 T5794 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2
[junit4:junit4] 2> 2856274 T5794 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index/
[junit4:junit4] 2> 2856275 T5794 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2856275 T5794 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index
[junit4:junit4] 2> 2856280 T5794 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2856281 T5794 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2856286 T5794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2856286 T5794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2856288 T5794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2856289 T5794 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2856290 T5794 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2856290 T5794 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2856291 T5794 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2856292 T5794 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2856293 T5794 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2856318 T5794 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2856328 T5794 oass.SolrIndexSearcher.<init> Opening Searcher@6391b24a main
[junit4:junit4] 2> 2856329 T5794 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/tlog
[junit4:junit4] 2> 2856331 T5794 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2856331 T5794 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2856375 T5795 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6391b24a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2856381 T5794 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2856381 T5794 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51026 collection:collection1 shard:shard1
[junit4:junit4] 2> 2856920 T5794 oasc.ZkController.register We are http://127.0.0.1:51026/collection1/ and leader is http://127.0.0.1:51023/collection1/
[junit4:junit4] 2> 2856920 T5794 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51026
[junit4:junit4] 2> 2856921 T5794 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2856921 T5794 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1068 name=collection1 org.apache.solr.core.SolrCore@3ece46eb url=http://127.0.0.1:51026/collection1 node=127.0.0.1:51026_ C1068_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51026_, base_url=http://127.0.0.1:51026}
[junit4:junit4] 2> 2856922 T5796 C1068 P51026 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2856924 T5794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2856924 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2856925 T5796 C1068 P51026 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2856926 T5796 C1068 P51026 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2856927 T5735 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2856928 T5735 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2856929 T5796 C1068 P51026 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2856929 T5735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2856983 T5769 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51026__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2856985 T5735 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1069 name=collection1 org.apache.solr.core.SolrCore@5aeb113e url=http://127.0.0.1:51019/collection1 node=127.0.0.1:51019_ C1069_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51019_, base_url=http://127.0.0.1:51019, leader=true}
[junit4:junit4] 2> 2857023 T5753 C1069 P51019 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@77b08ea0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@634c90a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2857025 T5753 C1069 P51019 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2857030 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10001 (1438211658155556864)]} 0 19
[junit4:junit4] 2> 2857031 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438211658145071104)]} 0 21
[junit4:junit4] 2> ASYNC NEW_CORE C1070 name=collection1 org.apache.solr.core.SolrCore@c659735 url=http://127.0.0.1:51023/collection1 node=127.0.0.1:51023_ C1070_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51023_, base_url=http://127.0.0.1:51023, leader=true}
[junit4:junit4] 2> 2857077 T5770 C1070 P51023 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c16266f lockFactory=org.apache.lucene.store.NativeFSLockFactory@43141ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2857078 T5770 C1070 P51023 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2857083 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438211658211131392)]} 0 11
[junit4:junit4] 2> 2857085 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10001 (1438211658210082816)]} 0 14
[junit4:junit4] 2> 2857113 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438211658250977280)]} 0 2
[junit4:junit4] 2> 2857114 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10002 (1438211658252025856)]} 0 2
[junit4:junit4] 2> 2857120 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438211658259365889)]} 0 1
[junit4:junit4] 2> 2857121 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10002 (1438211658259365888)]} 0 2
[junit4:junit4] 2> 2857196 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438211658339057664)]} 0 1
[junit4:junit4] 2> 2857199 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10003 (1438211658342203392)]} 0 1
[junit4:junit4] 2> 2857209 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438211658352689152)]} 0 2
[junit4:junit4] 2> 2857209 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10003 (1438211658352689153)]} 0 1
[junit4:junit4] 2> 2857278 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[2 (-1438211658425040896)]} 0 1
[junit4:junit4] 2> 2857280 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10002 (-1438211658428186624)]} 0 0
[junit4:junit4] 2> 2857287 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10002 (-1438211658435526656)]} 0 0
[junit4:junit4] 2> 2857288 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[2 (-1438211658436575232)]} 0 0
[junit4:junit4] 2> 2857293 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438211658440769536)]} 0 1
[junit4:junit4] 2> 2857295 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10004 (1438211658442866688)]} 0 1
[junit4:junit4] 2> 2857299 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438211658447060992)]} 0 1
[junit4:junit4] 2> 2857306 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10004 (1438211658451255296)]} 0 5
[junit4:junit4] 2> 2857376 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1438211658527801344)]} 0 1
[junit4:junit4] 2> 2857381 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1438211658533044224)]} 0 1
[junit4:junit4] 2> 2857386 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10005 (1438211658538287104)]} 0 1
[junit4:junit4] 2> 2857390 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10005 (1438211658543529984)]} 0 1
[junit4:junit4] 2> 2857483 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1438211658639998976)]} 0 1
[junit4:junit4] 2> 2857488 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1438211658645241856)]} 0 1
[junit4:junit4] 2> 2857492 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10006 (1438211658649436160)]} 0 1
[junit4:junit4] 2> 2857497 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10006 (1438211658654679040)]} 0 1
[junit4:junit4] 2> 2857571 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[5 (-1438211658732273664)]} 0 1
[junit4:junit4] 2> 2857575 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[5 (-1438211658736467968)]} 0 1
[junit4:junit4] 2> 2857579 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10005 (-1438211658740662272)]} 0 1
[junit4:junit4] 2> 2857580 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1438211658742759424)]} 0 1
[junit4:junit4] 2> 2857587 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10005 (-1438211658750099456)]} 0 0
[junit4:junit4] 2> 2857593 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1438211658755342336)]} 0 2
[junit4:junit4] 2> 2857596 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10007 (1438211658758488064)]} 0 1
[junit4:junit4] 2> 2857601 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10007 (1438211658763730944)]} 0 1
[junit4:junit4] 2> 2857615 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2857617 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51026__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51026_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51026"}
[junit4:junit4] 2> 2857627 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2857630 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2857633 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2857637 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2857684 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438211658850762752)]} 0 1
[junit4:junit4] 2> 2857691 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10008 (1438211658858102784)]} 0 1
[junit4:junit4] 2> C1068_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51026_, base_url=http://127.0.0.1:51026}
[junit4:junit4] 2> 2857714 T5785 C1068 P51026 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2857715 T5785 C1068 P51026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2857719 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1438211658856005632)]} 0 9
[junit4:junit4] 2> 2857727 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438211658856005632)]} 0 39
[junit4:junit4] 2> 2857730 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10008 (1438211658865442816)]} 0 2
[junit4:junit4] 2> 2857731 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10008 (1438211658865442816)]} 0 34
[junit4:junit4] 2> 2857761 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[8 (-1438211658932551680)]} 0 1
[junit4:junit4] 2> 2857763 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10008 (-1438211658934648832)]} 0 0
[junit4:junit4] 2> 2857771 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[8 (-1438211658939891712)]} 0 0
[junit4:junit4] 2> 2857773 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10008 (-1438211658938843136)]} 0 1
[junit4:junit4] 2> 2857774 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[8 (-1438211658939891712)]} 0 6
[junit4:junit4] 2> 2857776 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10008 (-1438211658938843136)]} 0 9
[junit4:junit4] 2> 2857780 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1438211658951426048)]} 0 1
[junit4:junit4] 2> 2857782 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10009 (1438211658953523200)]} 0 1
[junit4:junit4] 2> 2857796 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1438211658964008960)]} 0 1
[junit4:junit4] 2> 2857798 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1438211658964008960)]} 0 8
[junit4:junit4] 2> 2857801 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10009 (1438211658964008961)]} 0 3
[junit4:junit4] 2> 2857803 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10009 (1438211658964008961)]} 0 13
[junit4:junit4] 2> 2857902 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1438211659080400896)]} 0 0
[junit4:junit4] 2> 2857907 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10010 (1438211659084595200)]} 0 1
[junit4:junit4] 2> 2857913 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1438211659086692352)]} 0 1
[junit4:junit4] 2> 2857915 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10010 (1438211659089838080)]} 0 0
[junit4:junit4] 2> 2857916 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1438211659086692352)]} 0 8
[junit4:junit4] 2> 2857918 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10010 (1438211659089838080)]} 0 8
[junit4:junit4] 2> 2857976 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1438211659156946944)]} 0 1
[junit4:junit4] 2> 2857984 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10011 (1438211659165335552)]} 0 1
[junit4:junit4] 2> 2857985 T5769 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51026__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 2857986 T5769 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51026__collection1&state=recovering&nodeName=127.0.0.1:51026_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 2857995 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1438211659170578432)]} 0 1
[junit4:junit4] 2> 2857997 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1438211659170578432)]} 0 9
[junit4:junit4] 2> 2858005 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10011 (1438211659172675584)]} 0 6
[junit4:junit4] 2> 2858007 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10011 (1438211659172675584)]} 0 18
[junit4:junit4] 2> 2858009 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10 (-1438211659192598528)]} 0 0
[junit4:junit4] 2> 2858019 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10010 (-1438211659203084288)]} 0 0
[junit4:junit4] 2> 2858022 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10 (-1438211659199938560)]} 0 0
[junit4:junit4] 2> 2858026 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10 (-1438211659199938560)]} 0 13
[junit4:junit4] 2> 2858029 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10010 (-1438211659208327168)]} 0 0
[junit4:junit4] 2> 2858031 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10010 (-1438211659208327168)]} 0 7
[junit4:junit4] 2> 2858036 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1438211659218812928)]} 0 3
[junit4:junit4] 2> 2858038 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10012 (1438211659219861504)]} 0 3
[junit4:junit4] 2> 2858047 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1438211659227201536)]} 0 1
[junit4:junit4] 2> 2858053 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10012 (1438211659229298688)]} 0 1
[junit4:junit4] 2> 2858054 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10012 (1438211659229298688)]} 0 10
[junit4:junit4] 2> 2858060 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1438211659227201536)]} 0 18
[junit4:junit4] 2> 2858129 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10012 (-1438211659317379072)]} 0 2
[junit4:junit4] 2> 2858141 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[12 (-1438211659331010560)]} 0 1
[junit4:junit4] 2> 2858143 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10012 (-1438211659328913408)]} 0 1
[junit4:junit4] 2> 2858146 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10012 (-1438211659328913408)]} 0 7
[junit4:junit4] 2> 2858152 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10013 (1438211659341496320)]} 0 1
[junit4:junit4] 2> 2858154 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[12 (-1438211659338350592)]} 0 1
[junit4:junit4] 2> 2858155 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[12 (-1438211659338350592)]} 0 7
[junit4:junit4] 2> 2858160 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438211659349884928)]} 0 1
[junit4:junit4] 2> 2858169 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10013 (1438211659353030656)]} 0 1
[junit4:junit4] 2> 2858172 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1438211659355127808)]} 0 1
[junit4:junit4] 2> 2858173 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10013 (1438211659353030656)]} 0 11
[junit4:junit4] 2> 2858175 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438211659355127808)]} 0 11
[junit4:junit4] 2> 2858202 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10014 (1438211659393925120)]} 0 2
[junit4:junit4] 2> 2858203 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1438211659394973696)]} 0 2
[junit4:junit4] 2> 2858214 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10014 (1438211659402313728)]} 0 0
[junit4:junit4] 2> 2858217 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10014 (1438211659402313728)]} 0 9
[junit4:junit4] 2> 2858220 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1438211659405459456)]} 0 1
[junit4:junit4] 2> 2858222 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1438211659405459456)]} 0 10
[junit4:junit4] 2> 2858239 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10015 (1438211659432722432)]} 0 1
[junit4:junit4] 2> 2858244 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1438211659437965312)]} 0 1
[junit4:junit4] 2> 2858250 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10015 (1438211659441111040)]} 0 1
[junit4:junit4] 2> 2858252 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10015 (1438211659441111040)]} 0 6
[junit4:junit4] 2> 2858258 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1438211659449499648)]} 0 1
[junit4:junit4] 2> 2858260 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1438211659449499648)]} 0 6
[junit4:junit4] 2> 2858288 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10014 (-1438211659484102656)]} 0 1
[junit4:junit4] 2> 2858295 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10014 (-1438211659489345536)]} 0 1
[junit4:junit4] 2> 2858296 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10014 (-1438211659489345536)]} 0 4
[junit4:junit4] 2> 2858302 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10016 (1438211659498782720)]} 0 1
[junit4:junit4] 2> 2858306 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[14 (-1438211659504025600)]} 0 0
[junit4:junit4] 2> 2858318 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[14 (-1438211659513462784)]} 0 0
[junit4:junit4] 2> 2858320 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10016 (1438211659504025600)]} 0 4
[junit4:junit4] 2> 2858321 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10016 (1438211659504025600)]} 0 16
[junit4:junit4] 2> 2858323 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[14 (-1438211659513462784)]} 0 8
[junit4:junit4] 2> 2858327 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1438211659524997120)]} 0 1
[junit4:junit4] 2> 2858341 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1438211659532337152)]} 0 1
[junit4:junit4] 2> 2858343 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1438211659532337152)]} 0 12
[junit4:junit4] 2> 2858387 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10015 (-1438211659587911680)]} 0 2
[junit4:junit4] 2> 2858399 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10015 (-1438211659598397440)]} 0 1
[junit4:junit4] 2> 2858400 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10015 (-1438211659598397440)]} 0 5
[junit4:junit4] 2> 2858404 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[15 (-1438211659605737473)]} 0 1
[junit4:junit4] 2> 2858404 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10017 (1438211659605737472)]} 0 1
[junit4:junit4] 2> 2858411 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[15 (-1438211659609931776)]} 0 0
[junit4:junit4] 2> 2858414 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10017 (1438211659610980352)]} 0 1
[junit4:junit4] 2> 2858416 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[15 (-1438211659609931776)]} 0 9
[junit4:junit4] 2> 2858416 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10017 (1438211659610980352)]} 0 8
[junit4:junit4] 2> 2858422 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1438211659624611840)]} 0 1
[junit4:junit4] 2> 2858429 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1438211659628806144)]} 0 1
[junit4:junit4] 2> 2858430 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1438211659628806144)]} 0 5
[junit4:junit4] 2> 2858446 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10016 (-1438211659650826240)]} 0 0
[junit4:junit4] 2> 2858463 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[16 (-1438211659668652032)]} 0 0
[junit4:junit4] 2> 2858463 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10016 (-1438211659659214848)]} 0 0
[junit4:junit4] 2> 2858466 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10016 (-1438211659659214848)]} 0 13
[junit4:junit4] 2> 2858470 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10018 (1438211659674943488)]} 0 1
[junit4:junit4] 2> 2858471 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[16 (-1438211659673894912)]} 0 0
[junit4:junit4] 2> 2858472 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[16 (-1438211659673894912)]} 0 4
[junit4:junit4] 2> 2858476 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1438211659682283520)]} 0 0
[junit4:junit4] 2> 2858479 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10018 (1438211659680186368)]} 0 1
[junit4:junit4] 2> 2858481 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10018 (1438211659680186368)]} 0 7
[junit4:junit4] 2> 2858489 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438211659691720704)]} 0 1
[junit4:junit4] 2> 2858490 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1438211659691720704)]} 0 5
[junit4:junit4] 2> 2858520 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10019 (1438211659727372288)]} 0 1
[junit4:junit4] 2> 2858530 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1438211659732615168)]} 0 6
[junit4:junit4] 2> 2858543 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10019 (1438211659743100928)]} 0 2
[junit4:junit4] 2> 2858545 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438211659747295232)]} 0 3
[junit4:junit4] 2> 2858546 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1438211659747295232)]} 0 8
[junit4:junit4] 2> 2858547 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10019 (1438211659743100928)]} 0 13
[junit4:junit4] 2> 2858611 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10020 (1438211659823841280)]} 0 1
[junit4:junit4] 2> 2858620 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1438211659832229888)]} 0 1
[junit4:junit4] 2> 2858620 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10020 (1438211659828035584)]} 0 1
[junit4:junit4] 2> 2858622 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10020 (1438211659828035584)]} 0 7
[junit4:junit4] 2> 2858629 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438211659838521344)]} 0 1
[junit4:junit4] 2> 2858631 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1438211659838521344)]} 0 6
[junit4:junit4] 2> 2858652 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10021 (1438211659865784320)]} 0 1
[junit4:junit4] 2> 2858665 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1438211659879415808)]} 0 1
[junit4:junit4] 2> 2858669 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10021 (1438211659878367232)]} 0 2
[junit4:junit4] 2> 2858670 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10021 (1438211659878367232)]} 0 9
[junit4:junit4] 2> 2858673 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438211659884658688)]} 0 1
[junit4:junit4] 2> 2858674 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1438211659884658688)]} 0 6
[junit4:junit4] 2> 2858711 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10017 (-1438211659927650304)]} 0 1
[junit4:junit4] 2> 2858716 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[17 (-1438211659933941760)]} 0 0
[junit4:junit4] 2> 2858724 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10017 (-1438211659932893184)]} 0 0
[junit4:junit4] 2> 2858728 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10017 (-1438211659932893184)]} 0 13
[junit4:junit4] 2> 2858730 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[17 (-1438211659941281792)]} 0 1
[junit4:junit4] 2> 2858731 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[17 (-1438211659941281792)]} 0 8
[junit4:junit4] 2> 2858733 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10022 (1438211659950718976)]} 0 1
[junit4:junit4] 2> 2858735 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1438211659953864704)]} 0 1
[junit4:junit4] 2> 2858741 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10022 (1438211659954913280)]} 0 1
[junit4:junit4] 2> 2858745 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438211659960156160)]} 0 1
[junit4:junit4] 2> 2858746 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10022 (1438211659954913280)]} 0 10
[junit4:junit4] 2> 2858747 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1438211659960156160)]} 0 7
[junit4:junit4] 2> 2858775 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10023 (1438211659994759168)]} 0 1
[junit4:junit4] 2> 2858778 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438211659997904896)]} 0 1
[junit4:junit4] 2> 2858796 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438211660006293504)]} 0 2
[junit4:junit4] 2> 2858796 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10023 (1438211660003147776)]} 0 2
[junit4:junit4] 2> 2858797 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438211660006293504)]} 0 13
[junit4:junit4] 2> 2858798 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10023 (1438211660003147776)]} 0 16
[junit4:junit4] 2> 2858889 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1438211660114296832)]} 0 1
[junit4:junit4] 2> 2858893 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10024 (1438211660118491136)]} 0 1
[junit4:junit4] 2> 2858900 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438211660122685440)]} 0 1
[junit4:junit4] 2> 2858901 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1438211660122685440)]} 0 5
[junit4:junit4] 2> 2858904 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10024 (1438211660124782592)]} 0 0
[junit4:junit4] 2> 2858906 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10024 (1438211660124782592)]} 0 8
[junit4:junit4] 2> 2858948 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1438211660176162816)]} 0 1
[junit4:junit4] 2> 2858952 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10025 (1438211660180357120)]} 0 1
[junit4:junit4] 2> 2858960 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10025 (1438211660185600000)]} 0 1
[junit4:junit4] 2> 2858962 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1438211660183502848)]} 0 1
[junit4:junit4] 2> 2858963 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1438211660183502848)]} 0 9
[junit4:junit4] 2> 2858965 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10025 (1438211660185600000)]} 0 9
[junit4:junit4] 2> 2858996 T5735 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51026
[junit4:junit4] 2> 2858996 T5735 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1803924789
[junit4:junit4] 2> 2859025 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438211660257951744)]} 0 1
[junit4:junit4] 2> 2859030 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10026 (1438211660262146048)]} 0 2
[junit4:junit4] 2> 2859038 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10026 (1438211660267388928)]} 0 0
[junit4:junit4] 2> 2859040 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1438211660265291776)]} 0 1
[junit4:junit4] 2> 2859042 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438211660265291776)]} 0 9
[junit4:junit4] 2> 2859043 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10026 (1438211660267388928)]} 0 9
[junit4:junit4] 2> 2859077 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438211660311429120)]} 0 1
[junit4:junit4] 2> 2859085 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10027 (1438211660319817728)]} 0 1
[junit4:junit4] 2> 2859093 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10027 (1438211660325060608)]} 0 1
[junit4:junit4] 2> 2859095 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10027 (1438211660325060608)]} 0 7
[junit4:junit4] 2> 2859100 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1438211660327157760)]} 0 1
[junit4:junit4] 2> 2859101 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438211660327157760)]} 0 11
[junit4:junit4] 2> 2859148 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2859150 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51026__collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:51026_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51026"}
[junit4:junit4] 2> 2859158 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10018 (-1438211660397412352)]} 0 0
[junit4:junit4] 2> 2859164 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[18 (-1438211660403703808)]} 0 0
[junit4:junit4] 2> 2859170 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2859172 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2859172 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2859172 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2859175 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10018 (-1438211660401606656)]} 0 0
[junit4:junit4] 2> 2859178 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10018 (-1438211660401606656)]} 0 16
[junit4:junit4] 2> 2859185 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10028 (1438211660424675328)]} 0 1
[junit4:junit4] 2> C1068_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51026_, base_url=http://127.0.0.1:51026}
[junit4:junit4] 2> 2859190 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[18 (-1438211660415238144)]} 0 1
[junit4:junit4] 2> 2859195 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[18 (-1438211660415238144)]} 0 20
[junit4:junit4] 2> 2859200 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10028 (1438211660440403968)]} 0 1
[junit4:junit4] 2> 2859200 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1438211660441452544)]} 0 1
[junit4:junit4] 2> 2859207 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1438211660447744000)]} 0 1
[junit4:junit4] 2> 2859225 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10019 (-1438211660467666944)]} 0 0
[junit4:junit4] 2> 2859230 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10019 (-1438211660472909824)]} 0 0
[junit4:junit4] 2> 2859234 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[19 (-1438211660477104128)]} 0 1
[junit4:junit4] 2> 2859237 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10029 (1438211660479201280)]} 0 1
[junit4:junit4] 2> 2859239 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[19 (-1438211660482347008)]} 0 0
[junit4:junit4] 2> 2859242 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10029 (1438211660484444160)]} 0 1
[junit4:junit4] 2> 2859247 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438211660489687040)]} 0 1
[junit4:junit4] 2> 2859251 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438211660493881344)]} 0 1
[junit4:junit4] 2> 2859338 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10030 (1438211660584058880)]} 0 2
[junit4:junit4] 2> 2859341 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438211660588253184)]} 0 1
[junit4:junit4] 2> 2859344 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10030 (1438211660591398912)]} 0 1
[junit4:junit4] 2> 2859349 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438211660596641792)]} 0 1
[junit4:junit4] 2> 2859403 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10020 (-1438211660654313472)]} 0 0
[junit4:junit4] 2> 2859407 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[20 (-1438211660658507776)]} 0 0
[junit4:junit4] 2> 2859407 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10020 (-1438211660658507776)]} 0 0
[junit4:junit4] 2> 2859411 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[20 (-1438211660662702080)]} 0 1
[junit4:junit4] 2> 2859414 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10031 (1438211660664799232)]} 0 1
[junit4:junit4] 2> 2859417 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1438211660668993536)]} 0 1
[junit4:junit4] 2> 2859421 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10031 (1438211660672139264)]} 0 1
[junit4:junit4] 2> 2859424 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1438211660675284992)]} 0 1
[junit4:junit4] 2> 2859448 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10022 (-1438211660701499392)]} 0 1
[junit4:junit4] 2> 2859450 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[22 (-1438211660703596544)]} 0 0
[junit4:junit4] 2> 2859459 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[22 (-1438211660711985152)]} 0 3
[junit4:junit4] 2> 2859464 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10022 (-1438211660718276608)]} 0 1
[junit4:junit4] 2> 2859467 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1438211660720373760)]} 0 1
[junit4:junit4] 2> 2859469 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10032 (1438211660722470912)]} 0 2
[junit4:junit4] 2> 2859471 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1438211660724568064)]} 0 1
[junit4:junit4] 2> 2859473 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10032 (1438211660727713792)]} 0 1
[junit4:junit4] 2> 2859503 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1438211660758122496)]} 0 1
[junit4:junit4] 2> 2859508 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10033 (1438211660763365376)]} 0 1
[junit4:junit4] 2> 2859511 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1438211660766511104)]} 0 1
[junit4:junit4] 2> 2859513 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10033 (1438211660768608256)]} 0 1
[junit4:junit4] 2> 2859614 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[23 (-1438211660874514432)]} 0 1
[junit4:junit4] 2> 2859618 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[23 (-1438211660879757312)]} 0 1
[junit4:junit4] 2> 2859620 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10023 (-1438211660881854464)]} 0 0
[junit4:junit4] 2> 2859623 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1438211660883951616)]} 0 1
[junit4:junit4] 2> 2859626 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10023 (-1438211660888145920)]} 0 0
[junit4:junit4] 2> 2859629 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1438211660891291648)]} 0 1
[junit4:junit4] 2> 2859632 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10034 (1438211660893388800)]} 0 1
[junit4:junit4] 2> 2859636 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10034 (1438211660898631680)]} 0 0
[junit4:junit4] 2> 2859660 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1438211660922748928)]} 0 1
[junit4:junit4] 2> 2859665 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10035 (1438211660927991808)]} 0 1
[junit4:junit4] 2> 2859665 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1438211660927991808)]} 0 1
[junit4:junit4] 2> 2859670 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10035 (1438211660933234688)]} 0 1
[junit4:junit4] 2> 2859739 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[28 (-1438211661006635008)]} 0 0
[junit4:junit4] 2> 2859743 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10028 (-1438211661010829312)]} 0 0
[junit4:junit4] 2> 2859746 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[28 (-1438211661013975040)]} 0 1
[junit4:junit4] 2> 2859748 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10028 (-1438211661016072192)]} 0 0
[junit4:junit4] 2> 2859753 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10036 (1438211661020266496)]} 0 1
[junit4:junit4] 2> 2859755 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1438211661022363648)]} 0 1
[junit4:junit4] 2> 2859760 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10036 (1438211661027606528)]} 0 1
[junit4:junit4] 2> 2859761 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1438211661029703680)]} 0 1
[junit4:junit4] 2> 2859869 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[29 (-1438211661142949888)]} 0 0
[junit4:junit4] 2> 2859869 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10029 (-1438211661142949889)]} 0 0
[junit4:junit4] 2> 2859873 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[29 (-1438211661147144192)]} 0 0
[junit4:junit4] 2> 2859874 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10029 (-1438211661148192768)]} 0 1
[junit4:junit4] 2> 2859879 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1438211661151338496)]} 0 2
[junit4:junit4] 2> 2859883 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10037 (1438211661156581376)]} 0 2
[junit4:junit4] 2> 2859886 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1438211661159727104)]} 0 1
[junit4:junit4] 2> 2859890 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[31 (-1438211661164969984)]} 0 0
[junit4:junit4] 2> 2859894 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10037 (1438211661168115712)]} 0 1
[junit4:junit4] 2> 2859896 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[31 (-1438211661171261440)]} 0 0
[junit4:junit4] 2> 2859898 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10031 (-1438211661173358592)]} 0 0
[junit4:junit4] 2> 2859903 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1438211661178601472)]} 0 1
[junit4:junit4] 2> 2859905 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10031 (-1438211661180698624)]} 0 0
[junit4:junit4] 2> 2859908 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1438211661182795776)]} 0 1
[junit4:junit4] 2> 2859913 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10038 (1438211661188038656)]} 0 1
[junit4:junit4] 2> 2859917 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10038 (1438211661192232960)]} 0 1
[junit4:junit4] 2> 2859992 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51023/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2859993 T5796 C1068 P51026 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51026 START replicas=[http://127.0.0.1:51023/collection1/] nUpdates=100
[junit4:junit4] 2> 2859994 T5796 C1068 P51026 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2859995 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2859995 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2859995 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2859996 T5796 C1068 P51026 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/tlog/tlog.0000000000000000000 refcount=1}}
[junit4:junit4] 2> 2859996 T5796 C1068 P51026 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51023/collection1/. core=collection1
[junit4:junit4] 2> 2859997 T5796 C1068 P51026 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2859998 T5768 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2860023 T5772 C1070 P51023 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2860037 T5772 C1070 P51023 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c16266f lockFactory=org.apache.lucene.store.NativeFSLockFactory@43141ce7),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c16266f lockFactory=org.apache.lucene.store.NativeFSLockFactory@43141ce7),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2860038 T5772 C1070 P51023 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2860041 T5772 C1070 P51023 oass.SolrIndexSearcher.<init> Opening Searcher@6db53a0 realtime
[junit4:junit4] 2> 2860042 T5772 C1070 P51023 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2860043 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 2860045 T5796 C1068 P51026 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2860046 T5796 C1068 P51026 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2860048 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1438211661329596416)]} 0 1
[junit4:junit4] 2> 2860050 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10039 (1438211661331693568)]} 0 1
[junit4:junit4] 2> 2860062 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1438211661340082176)]} 0 5
[junit4:junit4] 2> 2860063 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10039 (1438211661345325056)]} 0 1
[junit4:junit4] 2> 2860064 T5770 C1070 P51023 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2860065 T5770 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 2860066 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2860067 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2860067 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2860071 T5767 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 2860072 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 5
[junit4:junit4] 2> 2860074 T5796 C1068 P51026 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index.20130618155842992
[junit4:junit4] 2> 2860074 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e41d2e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75e662b2) fullCopy=false
[junit4:junit4] 2> 2860078 T5772 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2860084 T5771 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2860091 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10040 (1438211661374685184)]} 0 1
[junit4:junit4] 2> 2860093 T5768 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 2860098 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1438211661382025216)]} 0 1
[junit4:junit4] 2> 2860100 T5769 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0_1.del&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2860108 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10040 (1438211661391462400)]} 0 2
[junit4:junit4] 2> 2860115 T5767 C1070 P51023 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2860117 T5796 C1068 P51026 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2860120 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1438211661402996736)]} 0 3
[junit4:junit4] 2> 2860127 T5796 C1068 P51026 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2860127 T5796 C1068 P51026 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2860128 T5796 C1068 P51026 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4] 2> 2860141 T5796 C1068 P51026 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2860142 T5796 C1068 P51026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2860149 T5796 C1068 P51026 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2860149 T5796 C1068 P51026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2860150 T5796 C1068 P51026 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2860153 T5796 C1068 P51026 oass.SolrIndexSearcher.<init> Opening Searcher@2cef0fb4 main
[junit4:junit4] 2> 2860155 T5795 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cef0fb4 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c40/16)}
[junit4:junit4] 2> 2860157 T5796 C1068 P51026 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index.20130618155842992 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index.20130618155842992;done=true>>]
[junit4:junit4] 2> 2860158 T5796 C1068 P51026 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index.20130618155842992
[junit4:junit4] 2> 2860158 T5796 C1068 P51026 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index.20130618155842992
[junit4:junit4] 2> 2860159 T5804 C1068 P51026 oasu.UpdateLog$LogReplayer.doReplay WARN Starting log replay tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/tlog/tlog.0000000000000000000 refcount=2} active=true starting pos=4491
[junit4:junit4] 2> 2860160 T5804 C1068 P51026 oasu.DirectUpdateHandler2.commit start commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2860161 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1438211661443891200)]} 0 7
[junit4:junit4] 2> 2860163 T5796 C1068 P51026 oasc.RecoveryStrategy.replay Replaying buffered documents. core=collection1
[junit4:junit4] 2> 2860166 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1438211661453328384)]} 0 1
[junit4:junit4] 2> 2860168 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10041 (1438211661455425536)]} 0 1
[junit4:junit4] 2> 2860169 T5804 C1068 P51026 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2860170 T5804 C1068 P51026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2860171 T5804 C1068 P51026 oass.SolrIndexSearcher.<init> Opening Searcher@24d52518 main
[junit4:junit4] 2> 2860172 T5804 C1068 P51026 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2860174 T5795 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24d52518 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c40/16)}
[junit4:junit4] 2> 2860175 T5804 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] {} 0 16
[junit4:junit4] 2> 2860175 T5804 C1068 P51026 oasu.UpdateLog$LogReplayer.run WARN Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=0 positionOfStart=4491}
[junit4:junit4] 2> 2860176 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2860176 T5796 C1068 P51026 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2860177 T5796 C1068 P51026 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2860180 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10041 (1438211661468008448)]} 0 1
[junit4:junit4] 2> 2860182 T5796 C1068 P51026 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2860272 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1438211661565526016)]} 0 1
[junit4:junit4] 2> 2860277 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1438211661569720320)]} 0 1
[junit4:junit4] 2> 2860284 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10042 (1438211661577060352)]} 0 1
[junit4:junit4] 2> 2860289 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10042 (1438211661582303232)]} 0 1
[junit4:junit4] 2> 2860330 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[33 (-1438211661626343424)]} 0 0
[junit4:junit4] 2> 2860335 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[33 (-1438211661631586304)]} 0 1
[junit4:junit4] 2> 2860339 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10033 (-1438211661635780609)]} 0 0
[junit4:junit4] 2> 2860339 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1438211661635780608)]} 0 1
[junit4:junit4] 2> 2860343 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10033 (-1438211661639974912)]} 0 0
[junit4:junit4] 2> 2860346 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1438211661642072064)]} 0 1
[junit4:junit4] 2> 2860351 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10043 (1438211661647314944)]} 0 1
[junit4:junit4] 2> 2860353 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[35 (-1438211661650460672)]} 0 0
[junit4:junit4] 2> 2860358 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10043 (1438211661654654976)]} 0 1
[junit4:junit4] 2> 2860360 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[35 (-1438211661657800704)]} 0 0
[junit4:junit4] 2> 2860365 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438211661663043584)]} 0 1
[junit4:junit4] 2> 2860370 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10035 (-1438211661668286464)]} 0 0
[junit4:junit4] 2> 2860382 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10035 (-1438211661680869376)]} 0 1
[junit4:junit4] 2> 2860391 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438211661689257984)]} 0 1
[junit4:junit4] 2> 2860393 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10044 (1438211661691355136)]} 0 1
[junit4:junit4] 2> 2860397 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10044 (1438211661695549440)]} 0 1
[junit4:junit4] 2> 2860419 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[36 (-1438211661718618112)]} 0 1
[junit4:junit4] 2> 2860423 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[36 (-1438211661723860992)]} 0 1
[junit4:junit4] 2> 2860426 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10036 (-1438211661725958144)]} 0 1
[junit4:junit4] 2> 2860429 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1438211661729103872)]} 0 1
[junit4:junit4] 2> 2860432 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10036 (-1438211661732249600)]} 0 1
[junit4:junit4] 2> 2860435 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1438211661735395328)]} 0 1
[junit4:junit4] 2> 2860447 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10045 (1438211661747978240)]} 0 2
[junit4:junit4] 2> 2860459 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10045 (1438211661760561152)]} 0 2
[junit4:junit4] 2> 2860493 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[37 (-1438211661797261312)]} 0 0
[junit4:junit4] 2> 2860497 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[37 (-1438211661801455616)]} 0 0
[junit4:junit4] 2> 2860502 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1438211661805649920)]} 0 1
[junit4:junit4] 2> 2860506 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1438211661809844224)]} 0 1
[junit4:junit4] 2> 2860520 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10037 (-1438211661825572864)]} 0 0
[junit4:junit4] 2> 2860524 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10037 (-1438211661829767168)]} 0 0
[junit4:junit4] 2> 2860529 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10046 (1438211661835010048)]} 0 1
[junit4:junit4] 2> 2860534 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10046 (1438211661839204352)]} 0 1
[junit4:junit4] 2> 2860579 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1438211661886390272)]} 0 1
[junit4:junit4] 2> 2860588 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1438211661895827456)]} 0 1
[junit4:junit4] 2> 2860607 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10047 (1438211661915750400)]} 0 1
[junit4:junit4] 2> 2860611 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10047 (1438211661919944704)]} 0 1
[junit4:junit4] 2> 2860618 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1438211661926236160)]} 0 2
[junit4:junit4] 2> 2860623 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1438211661932527616)]} 0 1
[junit4:junit4] 2> 2860641 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10048 (1438211661950353408)]} 0 2
[junit4:junit4] 2> 2860645 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10048 (1438211661956644864)]} 0 1
[junit4:junit4] 2> 2860671 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[39 (-1438211661983907840)]} 0 1
[junit4:junit4] 2> 2860679 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[39 (-1438211661992296448)]} 0 1
[junit4:junit4] 2> 2860683 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438211661995442176)]} 0 1
[junit4:junit4] 2> 2860688 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438211662000685056)]} 0 1
[junit4:junit4] 2> 2860694 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10039 (-1438211662008025088)]} 0 0
[junit4:junit4] 2> 2860698 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10039 (-1438211662012219392)]} 0 0
[junit4:junit4] 2> 2860703 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10049 (1438211662016413696)]} 0 1
[junit4:junit4] 2> 2860709 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2860711 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51026__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51026_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51026"}
[junit4:junit4] 2> 2860720 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10049 (1438211662031093760)]} 0 5
[junit4:junit4] 2> 2860730 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2860731 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2860731 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2860731 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2860795 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[41 (-1438211662112882688)]} 0 1
[junit4:junit4] 2> C1068_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51026_, base_url=http://127.0.0.1:51026}
[junit4:junit4] 2> 2860808 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[41 (-1438211662123368448)]} 0 1
[junit4:junit4] 2> 2860811 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {delete=[41 (-1438211662123368448)]} 0 7
[junit4:junit4] 2> 2860812 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[41]} 0 13
[junit4:junit4] 2> 2860816 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438211662134902784)]} 0 1
[junit4:junit4] 2> 2860820 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10041 (-1438211662140145664)]} 0 0
[junit4:junit4] 2> 2860827 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1438211662143291392)]} 0 1
[junit4:junit4] 2> 2860829 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438211662143291392)]} 0 6
[junit4:junit4] 2> 2860834 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10041 (-1438211662151680000)]} 0 0
[junit4:junit4] 2> 2860835 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {delete=[10041 (-1438211662151680000)]} 0 4
[junit4:junit4] 2> 2860837 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10041]} 0 12
[junit4:junit4] 2> 2860841 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10050 (1438211662161117184)]} 0 1
[junit4:junit4] 2> 2860854 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10050 (1438211662166360064)]} 0 5
[junit4:junit4] 2> 2860856 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10050 (1438211662166360064)]} 0 12
[junit4:junit4] 2> 2860858 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1438211662179991552)]} 0 1
[junit4:junit4] 2> 2860869 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1438211662187331584)]} 0 1
[junit4:junit4] 2> 2860870 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51026/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1438211662187331584)]} 0 6
[junit4:junit4] 2> 2860871 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51]} 0 9
[junit4:junit4] 2> 2860881 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10051 (1438211662203060224)]} 0 1
[junit4:junit4] 2> 2860895 T5786 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10051 (1438211662214594560)]} 0 0
[junit4:junit4] 2> 2860897 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10051 (1438211662214594560)]} 0 6
[junit4:junit4] 2> 2860905 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[42 (-1438211662229274624)]} 0 0
[junit4:junit4] 2> 2860912 T5785 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[42 (-1438211662233468928)]} 0 0
[junit4:junit4] 2> 2860913 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[42 (-1438211662233468928)]} 0 4
[junit4:junit4] 2> 2860922 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1438211662243954688)]} 0 4
[junit4:junit4] 2> 2860939 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10042 (-1438211662264926208)]} 0 1
[junit4:junit4] 2> 2860943 T5788 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1438211662263877632)]} 0 1
[junit4:junit4] 2> 2860945 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51026/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[52 (1438211662263877632)]} 0 8
[junit4:junit4] 2> 2860948 T5789 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10042 (-1438211662271217664)]} 0 1
[junit4:junit4] 2> 2860949 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10042 (-1438211662271217664)]} 0 4
[junit4:junit4] 2> 2860951 T5784 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52]} 0 25
[junit4:junit4] 2> 2860954 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10052 (1438211662280654848)]} 0 1
[junit4:junit4] 2> 2860963 T5787 C1068 P51026 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51023/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10052 (1438211662284849152)]} 0 2
[junit4:junit4] 2> 2860964 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10052 (1438211662284849152)]} 0 6
[junit4:junit4] 2> 2861003 T5735 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2861004 T5735 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2861007 T5735 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3ece46eb
[junit4:junit4] 2> 2861021 T5735 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=6,adds=6,deletesById=4,deletesByQuery=0,errors=0,cumulative_adds=46,cumulative_deletesById=20,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 2861022 T5735 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 2861023 T5735 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 2861023 T5735 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 2861032 T5735 C1068 P51026 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ed95c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3674105),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2861033 T5735 C1068 P51026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2861038 T5735 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 2861039 T5735 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 2861039 T5735 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2;done=false>>]
[junit4:junit4] 2> 2861040 T5735 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2
[junit4:junit4] 2> 2861040 T5735 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index;done=false>>]
[junit4:junit4] 2> 2861041 T5735 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index
[junit4:junit4] 2> 2861046 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2861057 T5757 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2861058 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2861058 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[44 (-1438211662389706752)]} 0 1
[junit4:junit4] 2> 2861062 T5777 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2861063 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2861064 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[44 (-1438211662395998208)]} 0 0
[junit4:junit4] 2> 2861067 T5764 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2861070 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438211662401241088)]} 0 1
[junit4:junit4] 2> 2861071 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10044 (-1438211662403338240)]} 0 0
[junit4:junit4] 2> 2861077 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438211662408581120)]} 0 1
[junit4:junit4] 2> 2861080 T5735 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 2861083 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10044 (-1438211662415921152)]} 0 0
[junit4:junit4] 2> 2861087 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10053 (1438211662420115456)]} 0 1
[junit4:junit4] 2> 2861092 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10053 (1438211662424309760)]} 0 1
[junit4:junit4] 2> 2861166 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438211662501904384)]} 0 1
[junit4:junit4] 2> 2861171 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438211662508195840)]} 0 1
[junit4:junit4] 2> 2861180 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10054 (1438211662516584448)]} 0 1
[junit4:junit4] 2> 2861190 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10054 (1438211662524973056)]} 0 3
[junit4:junit4] 2> 2861258 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438211662598373376)]} 0 1
[junit4:junit4] 2> 2861262 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438211662602567680)]} 0 1
[junit4:junit4] 2> 2861275 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10055 (1438211662617247744)]} 0 1
[junit4:junit4] 2> 2861280 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10055 (1438211662621442048)]} 0 1
[junit4:junit4] 2> 2861330 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1438211662673870848)]} 0 1
[junit4:junit4] 2> 2861337 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1438211662682259456)]} 0 1
[junit4:junit4] 2> 2861348 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10056 (1438211662692745216)]} 0 1
[junit4:junit4] 2> 2861353 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10056 (1438211662697988096)]} 0 1
[junit4:junit4] 2> 2861359 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[46 (-1438211662705328128)]} 0 0
[junit4:junit4] 2> 2861364 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[46 (-1438211662710571008)]} 0 0
[junit4:junit4] 2> 2861368 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1438211662714765312)]} 0 1
[junit4:junit4] 2> 2861373 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1438211662718959616)]} 0 2
[junit4:junit4] 2> 2861376 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10046 (-1438211662723153920)]} 0 0
[junit4:junit4] 2> 2861379 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10046 (-1438211662726299648)]} 0 0
[junit4:junit4] 2> 2861388 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10057 (1438211662735736832)]} 0 1
[junit4:junit4] 2> 2861393 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10057 (1438211662739931136)]} 0 1
[junit4:junit4] 2> 2861471 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438211662821720064)]} 0 1
[junit4:junit4] 2> 2861479 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438211662831157248)]} 0 1
[junit4:junit4] 2> 2861487 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10058 (1438211662838497280)]} 0 1
[junit4:junit4] 2> 2861492 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10058 (1438211662843740160)]} 0 1
[junit4:junit4] 2> 2861552 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[47 (-1438211662907703296)]} 0 0
[junit4:junit4] 2> 2861557 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[47 (-1438211662911897600)]} 0 1
[junit4:junit4] 2> 2861562 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438211662917140480)]} 0 1
[junit4:junit4] 2> 2861563 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10047 (-1438211662919237632)]} 0 0
[junit4:junit4] 2> 2861567 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10047 (-1438211662923431936)]} 0 0
[junit4:junit4] 2> 2861570 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438211662925529088)]} 0 2
[junit4:junit4] 2> 2861572 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10059 (1438211662928674816)]} 0 0
[junit4:junit4] 2> 2861579 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10059 (1438211662933917696)]} 0 2
[junit4:junit4] 2> 2861636 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[49 (-1438211662995783680)]} 0 0
[junit4:junit4] 2> 2861642 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[49 (-1438211663001026560)]} 0 1
[junit4:junit4] 2> 2861645 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10049 (-1438211663005220864)]} 0 0
[junit4:junit4] 2> 2861652 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438211663008366592)]} 0 5
[junit4:junit4] 2> 2861657 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10049 (-1438211663016755200)]} 0 1
[junit4:junit4] 2> 2861658 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438211663017803776)]} 0 1
[junit4:junit4] 2> 2861661 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10060 (1438211663020949504)]} 0 1
[junit4:junit4] 2> 2861668 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10060 (1438211663028289536)]} 0 1
[junit4:junit4] 2> 2861677 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[53 (-1438211663038775296)]} 0 0
[junit4:junit4] 2> 2861681 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[53 (-1438211663042969600)]} 0 0
[junit4:junit4] 2> 2861684 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10053 (-1438211663045066752)]} 0 1
[junit4:junit4] 2> 2861687 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1438211663048212480)]} 0 1
[junit4:junit4] 2> 2861690 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10053 (-1438211663051358208)]} 0 1
[junit4:junit4] 2> 2861693 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1438211663055552512)]} 0 1
[junit4:junit4] 2> 2861696 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10061 (1438211663058698240)]} 0 1
[junit4:junit4] 2> 2861701 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10061 (1438211663062892544)]} 0 1
[junit4:junit4] 2> 2861723 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1438211663085961216)]} 0 1
[junit4:junit4] 2> 2861730 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1438211663093301248)]} 0 2
[junit4:junit4] 2> 2861731 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10062 (1438211663094349824)]} 0 1
[junit4:junit4] 2> 2861735 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10062 (1438211663098544128)]} 0 1
[junit4:junit4] 2> 2861818 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1438211663185575936)]} 0 1
[junit4:junit4] 2> 2861823 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1438211663190818816)]} 0 1
[junit4:junit4] 2> 2861826 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10063 (1438211663193964544)]} 0 1
[junit4:junit4] 2> 2861830 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10063 (1438211663199207424)]} 0 0
[junit4:junit4] 2> 2861908 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[55 (-1438211663280996352)]} 0 0
[junit4:junit4] 2> 2861913 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[55 (-1438211663286239232)]} 0 1
[junit4:junit4] 2> 2861925 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10055 (-1438211663296724992)]} 0 2
[junit4:junit4] 2> 2861929 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1438211663301967872)]} 0 1
[junit4:junit4] 2> 2861931 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10055 (-1438211663305113600)]} 0 0
[junit4:junit4] 2> 2861933 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1438211663306162176)]} 0 1
[junit4:junit4] 2> 2861936 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10064 (1438211663309307904)]} 0 2
[junit4:junit4] 2> 2861941 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10064 (1438211663314550784)]} 0 1
[junit4:junit4] 2> 2861994 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1438211663370125312)]} 0 1
[junit4:junit4] 2> 2861999 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1438211663375368192)]} 0 1
[junit4:junit4] 2> 2862000 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10065 (1438211663376416768)]} 0 1
[junit4:junit4] 2> 2862004 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10065 (1438211663380611072)]} 0 1
[junit4:junit4] 2> 2862030 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1438211663407874048)]} 0 1
[junit4:junit4] 2> 2862035 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10066 (1438211663413116928)]} 0 1
[junit4:junit4] 2> 2862038 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1438211663417311232)]} 0 1
[junit4:junit4] 2> 2862041 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10066 (1438211663419408384)]} 0 1
[junit4:junit4] 2> 2862113 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1438211663494905856)]} 0 1
[junit4:junit4] 2> 2862117 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10067 (1438211663499100160)]} 0 1
[junit4:junit4] 2> 2862126 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1438211663503294464)]} 0 7
[junit4:junit4] 2> 2862129 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10067 (1438211663510634496)]} 0 2
[junit4:junit4] 2> 2862199 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10068 (1438211663585083392)]} 0 1
[junit4:junit4] 2> 2862199 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1438211663585083393)]} 0 1
[junit4:junit4] 2> 2862205 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10068 (1438211663592423424)]} 0 0
[junit4:junit4] 2> 2862206 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1438211663592423425)]} 0 1
[junit4:junit4] 2> 2862246 T5758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2862247 T5758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51026__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51026_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51026"}
[junit4:junit4] 2> 2862263 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2862263 T5764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2862263 T5757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2862270 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10057 (-1438211663660580864)]} 0 0
[junit4:junit4] 2> 2862271 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[57 (-1438211663660580865)]} 0 1
[junit4:junit4] 2> 2862281 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10057 (-1438211663672115200)]} 0 0
[junit4:junit4] 2> 2862283 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[57 (-1438211663674212352)]} 0 1
[junit4:junit4] 2> 2862287 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10069 (1438211663677358080)]} 0 1
[junit4:junit4] 2> 2862289 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69 (1438211663679455232)]} 0 1
[junit4:junit4] 2> 2862293 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69 (1438211663683649536)]} 0 1
[junit4:junit4] 2> 2862296 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10069 (1438211663687843840)]} 0 1
[junit4:junit4] 2> 2862335 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1438211663727689728)]} 0 2
[junit4:junit4] 2> 2862336 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10070 (1438211663728738304)]} 0 1
[junit4:junit4] 2> 2862339 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1438211663732932608)]} 0 0
[junit4:junit4] 2> 2862340 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10070 (1438211663732932609)]} 0 1
[junit4:junit4] 2> 2862348 T5793 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2862349 T5793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2862349 T5793 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2862405 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1438211663801090049)]} 0 1
[junit4:junit4] 2> 2862406 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10071 (1438211663801090048)]} 0 2
[junit4:junit4] 2> 2862410 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1438211663806332928)]} 0 1
[junit4:junit4] 2> 2862412 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10071 (1438211663808430080)]} 0 1
[junit4:junit4] 2> 2862462 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10072 (1438211663860858880)]} 0 1
[junit4:junit4] 2> 2862462 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1438211663860858881)]} 0 1
[junit4:junit4] 2> 2862470 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10072 (1438211663869247488)]} 0 1
[junit4:junit4] 2> 2862470 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1438211663869247489)]} 0 1
[junit4:junit4] 2> 2862509 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10073 (1438211663910141952)]} 0 1
[junit4:junit4] 2> 2862513 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[73 (1438211663914336256)]} 0 1
[junit4:junit4] 2> 2862516 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10073 (1438211663917481984)]} 0 1
[junit4:junit4] 2> 2862523 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[73 (1438211663920627712)]} 0 6
[junit4:junit4] 2> 2862554 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10058 (-1438211663957327872)]} 0 1
[junit4:junit4] 2> 2862557 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10058 (-1438211663961522176)]} 0 0
[junit4:junit4] 2> 2862563 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10074 (1438211663966765056)]} 0 1
[junit4:junit4] 2> 2862566 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[58 (-1438211663970959360)]} 0 1
[junit4:junit4] 2> 2862569 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10074 (1438211663973056512)]} 0 1
[junit4:junit4] 2> 2862573 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[58 (-1438211663978299392)]} 0 0
[junit4:junit4] 2> 2862578 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[74 (1438211663982493696)]} 0 2
[junit4:junit4] 2> 2862582 T5769 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[74 (1438211663987736576)]} 0 1
[junit4:junit4] 2> 2862647 T5751 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10059 (-1438211664055894016)]} 0 1
[junit4:junit4] 2> 2862650 T5770 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10059 (-1438211664059039744)]} 0 0
[junit4:junit4] 2> 2862653 T5753 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10075 (1438211664062185472)]} 0 0
[junit4:junit4] 2> 2862658 T5772 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10075 (1438211664066379776)]} 0 1
[junit4:junit4] 2> 2862660 T5749 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[59 (-1438211664069525504)]} 0 1
[junit4:junit4] 2> 2862663 T5767 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[59 (-1438211664072671232)]} 0 0
[junit4:junit4] 2> 2862668 T5752 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[75 (1438211664076865536)]} 0 1
[junit4:junit4] 2> 2862678 T5768 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[75 (1438211664087351296)]} 0 1
[junit4:junit4] 2> 2862678 T5750 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10076 (1438211664087351296)]} 0 1
[junit4:junit4] 2> 2862687 T5771 C1070 P51023 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10076 (1438211664096788480)]} 0 1
[junit4:junit4] 2> 2862701 T5748 C1069 P51019 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[76 (1438211664111468544)]} 0 1
[junit4:junit4] 2> 2862710 T5769 C1070 P51023 oasup.LogUpdatePr
[...truncated too long message...]
=43,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=35,cumulative_deletesById=19,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 2962763 T5735 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 2962764 T5735 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 2962764 T5735 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 2962767 T5735 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 2962768 T5735 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 2962769 T5735 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2;done=false>>]
[junit4:junit4] 2> 2962769 T5735 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2
[junit4:junit4] 2> 2962770 T5735 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index;done=false>>]
[junit4:junit4] 2> 2962770 T5735 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371585502145/jetty2/index
[junit4:junit4] 2> 2962772 T5852 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89888227469426698-127.0.0.1:51026_-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 2962777 T5818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2962791 T5735 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 2962873 T5735 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 2962879 T5735 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51016 51016
[junit4:junit4] 2> 2962952 T5777 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2963260 T5777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2963261 T5777 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch -Dtests.seed=D4A1FB632969ACEC -Dtests.slow=true -Dtests.locale=ar_TN -Dtests.timezone=America/Anguilla -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 124s | RecoveryZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: expected:<179> but was:<33>
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D4A1FB632969ACEC:5547757B5E36CCD0]:0)
[junit4:junit4] > at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2963324 T5735 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 124115 T5734 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 2964078 T5818 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2964078 T5818 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), text=PostingsFormat(name=Direct), _version_=PostingsFormat(name=TestBloomFilteredLucene41Postings), rnd_b=MockFixedIntBlock(blockSize=884), intDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), timestamp=PostingsFormat(name=TestBloomFilteredLucene41Postings), id=MockFixedIntBlock(blockSize=884), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), range_facet_sl=PostingsFormat(name=Direct), range_facet_si=Pulsing41(freqCutoff=12 minBlockSize=41 maxBlockSize=162), other_tl1=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=Pulsing41(freqCutoff=12 minBlockSize=41 maxBlockSize=162), a_si=Pulsing41(freqCutoff=12 minBlockSize=41 maxBlockSize=162)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=ar_TN, timezone=America/Anguilla
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=3,free=272959240,total=492830720
[junit4:junit4] 2> NOTE: All tests run in this JVM: [AutoCommitTest, TestDynamicFieldCollectionResource, LoggingHandlerTest, TestRandomDVFaceting, SortByFunctionTest, TestSystemIdResolver, TestUniqueKeyFieldResource, TestDocumentBuilder, TestCollationField, AlternateDirectoryTest, TestSolrCoreProperties, IndexBasedSpellCheckerTest, ClusterStateTest, TestHashPartitioner, TestMultiCoreConfBootstrap, ResponseLogComponentTest, ZkSolrClientTest, TestQueryTypes, FastVectorHighlighterTest, TestSolrXMLSerializer, SuggesterTSTTest, TestComponentsName, ZkControllerTest, TestRTGBase, SpatialFilterTest, TestClassNameShortening, TestExtendedDismaxParser, TestSolrQueryParserResource, DirectUpdateHandlerTest, TestRangeQuery, TestPropInjectDefaults, IndexSchemaRuntimeFieldTest, ShowFileRequestHandlerTest, TestSolrDeletionPolicy1, PeerSyncTest, TestPostingsSolrHighlighter, SolrPluginUtilsTest, BasicDistributedZkTest, PingRequestHandlerTest, DebugComponentTest, SolrCmdDistributorTest, TestFastOutputStream, PreAnalyzedUpdateProcessorTest, TestFaceting, TestCopyFieldCollectionResource, QueryEqualityTest, UUIDFieldTest, ZkNodePropsTest, FileUtilsTest, DisMaxRequestHandlerTest, EchoParamsTest, TestLFUCache, UpdateParamsTest, PluginInfoTest, SolrRequestParserTest, TestShardHandlerFactory, SearchHandlerTest, RegexBoostProcessorTest, DateMathParserTest, ShardRoutingCustomTest, SyncSliceTest, DateFieldTest, TestFieldResource, TestConfig, TestFastLRUCache, BasicZkTest, TestDistributedGrouping, TestRealTimeGet, AnalysisAfterCoreReloadTest, TestWordDelimiterFilterFactory, TestReplicationHandler, TestFieldCollectionResource, TermVectorComponentDistributedTest, TestManagedSchema, TestStressReorder, BadComponentTest, IndexSchemaTest, LeaderElectionIntegrationTest, LegacyHTMLStripCharFilterTest, SuggesterWFSTTest, CachingDirectoryFactoryTest, BasicDistributedZk2Test, SolrCoreCheckLockOnStartupTest, ReturnFieldsTest, TestSolrXmlPersistence, TestJoin, ClusterStateUpdateTest, UnloadDistributedZkTest, TestPseudoReturnFields, TestFunctionQuery, StandardRequestHandlerTest, RequestHandlersTest, SpellingQueryConverterTest, IndexReaderFactoryTest, HighlighterConfigTest, TestSchemaVersionResource, TestBinaryResponseWriter, SignatureUpdateProcessorFactoryTest, TestPropInject, PrimUtilsTest, TestArbitraryIndexDir, LukeRequestHandlerTest, SampleTest, TestRandomFaceting, TestDynamicFieldResource, NoCacheHeaderTest, ShardSplitTest, TestCSVLoader, OpenExchangeRatesOrgProviderTest, SolrInfoMBeanTest, LeaderElectionTest, TestNumberUtils, SolrCoreTest, NumericFieldsTest, TestCoreContainer, TestBM25SimilarityFactory, TestCloudManagedSchemaAddField, XmlUpdateRequestHandlerTest, TestPartialUpdateDeduplication, CollectionsAPIDistributedZkTest, TestFieldTypeResource, TestXIncludeConfig, DistributedSpellCheckComponentTest, ChaosMonkeyNothingIsSafeTest, TestIndexSearcher, TestSolrDeletionPolicy2, ScriptEngineTest, TestLMDirichletSimilarityFactory, SliceStateUpdateTest, TestZkChroot, TestSchemaResource, TestSolr4Spatial, DistributedTermsComponentTest, TestUpdate, TestAtomicUpdateErrorCases, TestWriterPerf, OpenCloseCoreStressTest, TestAddFieldRealTimeGet, PathHierarchyTokenizerFactoryTest, ChaosMonkeySafeLeaderTest, OverseerTest, ShardRoutingTest, RecoveryZkTest]
[junit4:junit4] Completed in 124.96s, 1 test, 1 failure <<< FAILURES!
[...truncated 451 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1250 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 130 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 579 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/579/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
2 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
collection already exists: awholynewcollection_0
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: awholynewcollection_0
at __randomizedtesting.SeedInfo.seed([C44103AB69D2E3:8122CF1BDC36B2DF]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:264)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:313)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1455)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:438)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:49525/xm_bai/y/collection1lastClient and got 267 from http://127.0.0.1:49532/xm_bai/y/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49525/xm_bai/y/collection1lastClient and got 267 from http://127.0.0.1:49532/xm_bai/y/collection1
at __randomizedtesting.SeedInfo.seed([C44103AB69D2E3:8122CF1BDC36B2DF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9231 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 2> 645744 T785 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /hm_dlx/z
[junit4:junit4] 2> 645787 T785 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1371714255860
[junit4:junit4] 2> 645790 T785 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 645791 T786 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 645807 T785 oasc.ZkTestServer.run start zk server on port:49392
[junit4:junit4] 2> 645814 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 645854 T792 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cc05349 name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 645855 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 645855 T785 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 645893 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 645900 T794 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17deda19 name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 645900 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 645901 T785 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 645914 T785 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 645925 T785 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 645939 T785 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 645948 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 645949 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 645964 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 645965 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 645977 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 645979 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 645992 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 645999 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 646007 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 646008 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 646020 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 646021 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 646030 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 646031 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 646041 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 646042 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 646050 T785 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 646051 T785 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 646670 T785 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 646696 T785 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49395
[junit4:junit4] 2> 646697 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 646698 T785 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 646698 T785 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139
[junit4:junit4] 2> 646699 T785 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/solr.xml
[junit4:junit4] 2> 646700 T785 oasc.CoreContainer.<init> New CoreContainer 821461936
[junit4:junit4] 2> 646700 T785 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/'
[junit4:junit4] 2> 646701 T785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/'
[junit4:junit4] 2> 646889 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139
[junit4:junit4] 2> 646892 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/collection1
[junit4:junit4] 2> 646893 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/collection1/conf
[junit4:junit4] 2> 647088 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/collection1/conf/xslt
[junit4:junit4] 2> 647092 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/collection1/lib
[junit4:junit4] 2> 647099 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/collection1/lib/classes
[junit4:junit4] 2> 647102 T785 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/conf/core.properties, adding to cores
[junit4:junit4] 2> 647118 T785 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 647121 T785 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 647125 T785 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 647126 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 647127 T785 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 647127 T785 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 647128 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 647128 T785 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 647129 T785 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 647129 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 647159 T785 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 647160 T785 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49392/solr
[junit4:junit4] 2> 647160 T785 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 647171 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 647174 T805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@733ef04 name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 647174 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 647182 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 647213 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 647217 T807 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f412d31 name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 647238 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 647255 T785 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 647271 T785 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 647282 T785 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 647291 T785 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 647295 T785 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 647307 T785 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 647325 T785 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 647337 T785 oasc.Overseer.start Overseer (id=89896665472696323-127.0.0.1:49395_hm_dlx%2Fz-n_0000000000) starting
[junit4:junit4] 2> 647349 T785 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 647363 T809 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 647367 T785 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 647379 T785 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 647387 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 647406 T808 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 647409 T785 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 647410 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 647414 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 647443 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 647446 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 647453 T812 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fb79bf1 name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 647453 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 647463 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 647473 T785 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 648043 T785 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 648052 T785 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49399
[junit4:junit4] 2> 648053 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 648054 T785 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 648054 T785 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544
[junit4:junit4] 2> 648055 T785 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/solr.xml
[junit4:junit4] 2> 648056 T785 oasc.CoreContainer.<init> New CoreContainer 1376401576
[junit4:junit4] 2> 648057 T785 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/'
[junit4:junit4] 2> 648057 T785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/'
[junit4:junit4] 2> 648239 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544
[junit4:junit4] 2> 648241 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/collection1
[junit4:junit4] 2> 648242 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/collection1/conf
[junit4:junit4] 2> 648437 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/collection1/conf/xslt
[junit4:junit4] 2> 648443 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/collection1/lib
[junit4:junit4] 2> 648445 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/collection1/lib/classes
[junit4:junit4] 2> 648448 T785 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/conf/core.properties, adding to cores
[junit4:junit4] 2> 648469 T785 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 648470 T785 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 648471 T785 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 648471 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 648472 T785 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 648472 T785 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 648473 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 648474 T785 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 648475 T785 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 648476 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 648502 T785 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 648503 T785 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49392/solr
[junit4:junit4] 2> 648503 T785 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 648506 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 648513 T823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53d7ac21 name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 648513 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 648521 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 648549 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 648555 T825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29783b83 name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 648556 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 648573 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 649587 T785 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 649592 T785 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 649608 T825 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 649616 T812 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 649617 T807 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 649625 T785 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 649626 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 649626 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 650218 T785 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 650228 T785 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49402
[junit4:junit4] 2> 650229 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 650230 T785 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 650231 T785 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723
[junit4:junit4] 2> 650231 T785 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/solr.xml
[junit4:junit4] 2> 650232 T785 oasc.CoreContainer.<init> New CoreContainer 2029497577
[junit4:junit4] 2> 650233 T785 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/'
[junit4:junit4] 2> 650233 T785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/'
[junit4:junit4] 2> 650413 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723
[junit4:junit4] 2> 650415 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/collection1
[junit4:junit4] 2> 650417 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/collection1/conf
[junit4:junit4] 2> 650600 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/collection1/conf/xslt
[junit4:junit4] 2> 650605 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/collection1/lib
[junit4:junit4] 2> 650607 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/collection1/lib/classes
[junit4:junit4] 2> 650610 T785 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/conf/core.properties, adding to cores
[junit4:junit4] 2> 650625 T785 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 650625 T785 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 650627 T785 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 650627 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 650628 T785 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 650629 T785 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 650630 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 650630 T785 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 650631 T785 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 650632 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 650664 T785 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 650665 T785 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49392/solr
[junit4:junit4] 2> 650665 T785 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 650668 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 650673 T837 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32bc46e name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 650674 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 650680 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 650708 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 650715 T839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5431939f name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 650716 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 650737 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 651750 T785 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 651755 T785 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 651768 T825 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 651768 T807 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 651773 T839 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 651781 T812 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 651788 T785 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 651789 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 651790 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 652399 T785 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 652406 T785 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49405
[junit4:junit4] 2> 652408 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 652408 T785 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 652409 T785 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887
[junit4:junit4] 2> 652409 T785 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/solr.xml
[junit4:junit4] 2> 652410 T785 oasc.CoreContainer.<init> New CoreContainer 1360569011
[junit4:junit4] 2> 652411 T785 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/'
[junit4:junit4] 2> 652411 T785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/'
[junit4:junit4] 2> 652656 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887
[junit4:junit4] 2> 652659 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/collection1
[junit4:junit4] 2> 652661 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/collection1/conf
[junit4:junit4] 2> 652862 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/collection1/conf/xslt
[junit4:junit4] 2> 652870 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/collection1/lib
[junit4:junit4] 2> 652875 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/collection1/lib/classes
[junit4:junit4] 2> 652878 T785 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/conf/core.properties, adding to cores
[junit4:junit4] 2> 652893 T785 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 652894 T785 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 652895 T785 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 652895 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 652896 T785 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 652896 T785 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 652897 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 652898 T785 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 652898 T785 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 652899 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 652930 T785 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 652931 T785 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49392/solr
[junit4:junit4] 2> 652932 T785 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 652936 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 652941 T851 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e1ad9e9 name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 652942 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 652949 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 652972 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 652979 T853 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4002bf5c name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 652979 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 653002 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 654015 T785 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 654021 T785 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 654036 T853 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 654039 T812 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 654042 T807 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 654043 T839 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 654044 T825 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 654057 T785 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 654058 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 654058 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 654641 T785 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 654654 T785 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49408
[junit4:junit4] 2> 654656 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 654657 T785 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 654658 T785 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161
[junit4:junit4] 2> 654658 T785 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/solr.xml
[junit4:junit4] 2> 654659 T785 oasc.CoreContainer.<init> New CoreContainer 896071197
[junit4:junit4] 2> 654660 T785 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/'
[junit4:junit4] 2> 654661 T785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/'
[junit4:junit4] 2> 654837 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161
[junit4:junit4] 2> 654839 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/collection1
[junit4:junit4] 2> 654841 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/collection1/conf
[junit4:junit4] 2> 655017 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/collection1/conf/xslt
[junit4:junit4] 2> 655023 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/collection1/lib
[junit4:junit4] 2> 655024 T785 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/collection1/lib/classes
[junit4:junit4] 2> 655027 T785 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/conf/core.properties, adding to cores
[junit4:junit4] 2> 655046 T785 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 655046 T785 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 655047 T785 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 655048 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 655048 T785 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 655049 T785 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 655049 T785 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 655050 T785 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 655051 T785 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 655059 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 655088 T785 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 655089 T785 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49392/solr
[junit4:junit4] 2> 655089 T785 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 655092 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 655099 T865 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48186010 name:ZooKeeperConnection Watcher:127.0.0.1:49392 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 655099 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 655107 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 655139 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 655146 T867 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48f0918a name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 655146 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 655164 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 656176 T785 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 656179 T785 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 656201 T812 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656202 T867 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656207 T807 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656216 T785 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 656216 T785 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 656217 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 656223 T825 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656224 T839 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656224 T853 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 656243 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 656272 T817 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 656279 T807 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 656283 T809 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":null,
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"nodes_used_collection",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 656283 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 656284 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 656285 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 656286 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 656305 T818 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 2> 656305 T862 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 2> 656306 T802 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 2> 656308 T846 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 2> 656511 T808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 656513 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 656513 T808 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4] 2> 656525 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 656535 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 656551 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 656569 T807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 656570 T825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 656572 T839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 656578 T853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 656582 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 656584 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 657315 T818 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 657318 T818 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 657317 T802 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 657320 T802 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 657316 T846 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 657321 T846 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 657316 T862 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 657322 T862 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 657326 T846 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 657326 T846 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 657329 T846 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 657330 T846 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 657338 T846 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 657343 T846 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 2> 657370 T818 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 657371 T818 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 657372 T862 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 657372 T862 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 657374 T802 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 657376 T802 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 657413 T802 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 657414 T818 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 657416 T862 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 2> 657506 T802 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 657506 T818 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 657514 T802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 2> 657515 T818 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 2> 657646 T846 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 657666 T862 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 657696 T802 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 657743 T818 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 657961 T846 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 657966 T846 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 657992 T846 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4] 2> 658017 T862 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 658050 T862 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 658059 T818 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 658064 T818 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 658101 T862 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4] 2> 658105 T818 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4] 2> 658147 T802 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 658160 T802 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 658239 T802 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4] 2> 661518 T862 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 661589 T846 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 661594 T862 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 661617 T862 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 661669 T846 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 661684 T846 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 661769 T802 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 661818 T802 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 661899 T802 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 661894 T846 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661878 T862 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661916 T862 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661935 T846 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661941 T862 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661936 T818 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 661971 T862 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 661972 T862 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 661973 T862 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661978 T846 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661984 T862 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 661985 T862 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 661985 T862 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 661986 T862 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 2> 661986 T862 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d3bd913
[junit4:junit4] 2> 661987 T862 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 661990 T846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662014 T846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662015 T846 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661994 T862 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 2> 662016 T862 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 2> 662016 T862 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 662017 T862 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 2> 662028 T818 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 662034 T846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662035 T846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662036 T846 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662037 T846 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 2> 662049 T846 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d3bd913
[junit4:junit4] 2> 662050 T846 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662062 T818 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 662066 T846 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 2> 662066 T846 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 2> 662067 T846 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 662079 T846 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 2> 662086 T862 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6058de27 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17446ad5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 662088 T862 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 662105 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 662106 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 662115 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 662116 T862 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 662118 T862 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 662118 T862 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 662119 T862 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 662121 T862 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 662122 T862 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 662148 T846 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c823d00 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1da3fa09),segFN=segments_1,generation=1}
[junit4:junit4] 2> 662150 T846 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 662156 T846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 662157 T846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 662158 T846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 662161 T846 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 662163 T846 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 662164 T846 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 662164 T846 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 662166 T846 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 662173 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 662175 T846 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 662208 T846 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 662211 T862 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 662257 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 662269 T846 oass.SolrIndexSearcher.<init> Opening Searcher@1c5bf96d main
[junit4:junit4] 2> 662271 T846 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 2> 662272 T846 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 662273 T846 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 662274 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 662277 T862 oass.SolrIndexSearcher.<init> Opening Searcher@6e59278 main
[junit4:junit4] 2> 662278 T862 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 2> 662286 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662286 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662287 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 662289 T862 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 662290 T862 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 662298 T874 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@6e59278 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 662303 T862 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 662303 T862 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:49408/hm_dlx/z collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 662306 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662307 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662307 T802 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662308 T802 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 2> 662309 T802 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d3bd913
[junit4:junit4] 2> 662310 T802 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662312 T873 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@1c5bf96d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 662314 T862 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 2> 662321 T802 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 2> 662322 T802 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 2> 662322 T802 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 662327 T846 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 662328 T846 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:49405/hm_dlx/z collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 662338 T802 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 2> 662340 T846 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 662349 T802 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25d50010 lockFactory=org.apache.lucene.store.NativeFSLockFactory@559dd738),segFN=segments_1,generation=1}
[junit4:junit4] 2> 662350 T802 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 662363 T818 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 662364 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 662371 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 662373 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 662374 T802 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 662375 T802 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 662376 T802 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 662377 T802 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 662377 T802 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 662378 T802 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 662424 T818 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 662446 T818 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 662449 T862 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 662470 T802 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 662482 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662483 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662484 T818 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 662483 T802 oass.SolrIndexSearcher.<init> Opening Searcher@324ff339 main
[junit4:junit4] 2> 662488 T802 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 2> 662489 T802 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 662490 T802 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 662482 T846 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 662501 T862 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179985
[junit4:junit4] 2> 662502 T875 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@324ff339 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 662505 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 662506 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 662506 T818 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662507 T818 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 2> 662507 T818 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d3bd913
[junit4:junit4] 2> 662508 T818 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 662521 T818 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 2> 662522 T818 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 2> 662523 T818 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 662525 T818 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 2> 662530 T802 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 662530 T802 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:49395/hm_dlx/z collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 662534 T846 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179997
[junit4:junit4] 2> 662536 T818 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516fc4ce lockFactory=org.apache.lucene.store.NativeFSLockFactory@4cc2e9fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 662537 T818 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 662543 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 662544 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 662545 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 662546 T818 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 662547 T818 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 662548 T818 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 662548 T818 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 662549 T818 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 662550 T818 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 662577 T818 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 662613 T818 oass.SolrIndexSearcher.<init> Opening Searcher@6b01cc7d main
[junit4:junit4] 2> 662615 T818 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 2> 662617 T818 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 662617 T818 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 662622 T876 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@6b01cc7d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 662630 T818 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 662630 T818 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:49399/hm_dlx/z collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 662680 T808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 662739 T807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 662740 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 662740 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 662739 T839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 662741 T853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 662742 T825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 663006 T862 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 663007 T862 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 663008 T862 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 663008 T862 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:49408/hm_dlx/z START replicas=[http://127.0.0.1:49399/hm_dlx/z/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 2> 663009 T862 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:49408/hm_dlx/z DONE. We have no versions. sync failed.
[junit4:junit4] 2> 663010 T862 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 663011 T862 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 663011 T862 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 663012 T862 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C327 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@150511ef url=http://127.0.0.1:49399/hm_dlx/z/nodes_used_collection_shard1_replica1 node=127.0.0.1:49399_hm_dlx%2Fz C327_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49399_hm_dlx%2Fz, base_url=http://127.0.0.1:49399/hm_dlx/z}
[junit4:junit4] 2> 663031 T819 C327 P49399 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/hm_dlx/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 663040 T846 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 663041 T846 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 663041 T846 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 663042 T846 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:49405/hm_dlx/z START replicas=[http://127.0.0.1:49395/hm_dlx/z/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4] 2> 663043 T846 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:49405/hm_dlx/z DONE. We have no versions. sync failed.
[junit4:junit4] 2> 663043 T846 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 663044 T846 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 663044 T846 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 663045 T846 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C328 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@43f420ba url=http://127.0.0.1:49395/hm_dlx/z/nodes_used_collection_shard2_replica1 node=127.0.0.1:49395_hm_dlx%2Fz C328_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49395_hm_dlx%2Fz, base_url=http://127.0.0.1:49395/hm_dlx/z}
[junit4:junit4] 2> 663077 T803 C328 P49395 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp=/hm_dlx/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 664290 T808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 664317 T807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664318 T825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664318 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664321 T839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664329 T853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664329 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 664358 T818 oasc.ZkController.register We are http://127.0.0.1:49399/hm_dlx/z/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 664358 T818 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:49399/hm_dlx/z
[junit4:junit4] 2> 664359 T818 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 664359 T818 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 664361 T879 C327 P49399 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 664361 T879 C327 P49399 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 664362 T879 C327 P49399 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4] 2> 664365 T818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 664369 T818 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8064
[junit4:junit4] 2> 664371 T879 C327 P49399 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 664381 T846 oasc.ZkController.register We are http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 664382 T846 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:49405/hm_dlx/z
[junit4:junit4] 2> 664382 T846 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 664383 T846 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4] 2> 664384 T862 oasc.ZkController.register We are http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 664385 T862 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:49408/hm_dlx/z
[junit4:junit4] 2> 664385 T862 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 664386 T862 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4] 2> 664386 T802 oasc.ZkController.register We are http://127.0.0.1:49395/hm_dlx/z/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 664387 T802 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:49395/hm_dlx/z
[junit4:junit4] 2> 664387 T802 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 664388 T802 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 664392 T880 C328 P49395 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 664393 T880 C328 P49395 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 664398 T880 C328 P49395 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4] 2> 664402 T802 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 664403 T880 C328 P49395 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 664409 T862 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 664411 T802 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8105
[junit4:junit4] 2> 664427 T846 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 664431 T846 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8123
[junit4:junit4] 2> 664432 T862 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8127
[junit4:junit4] 2> 664439 T809 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 2> 664500 T863 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49399_hm_dlx%2Fz_nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 664509 T848 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49395_hm_dlx%2Fz_nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 664510 T825 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 664519 T807 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 664523 T809 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=8064},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=8105},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=8123},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=8127},core=nodes_used_collection_shard1_replica2}}}
[junit4:junit4] 2> 664523 T817 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=8251
[junit4:junit4] 2> 664525 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 664531 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 664537 T882 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@158bb81c name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 664537 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 664542 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 664551 T785 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 664556 T785 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 664561 T884 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c23c135 name:ZooKeeperConnection Watcher:127.0.0.1:49392/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 664561 T785 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 664565 T785 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 664631 T819 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=5&maxShardsPerNode=6&name=awholynewcollection_0&replicationFactor=5&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 664636 T807 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 664640 T809 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "maxShardsPerNode":"6",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_0",
[junit4:junit4] 2> "replicationFactor":"5"}
[junit4:junit4] 2> 664640 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 664641 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 664641 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 664641 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica4 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 664641 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica5 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 664642 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 664644 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 664647 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 664652 T835 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=down
[junit4:junit4] 2> 664664 T849 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica5 state=down
[junit4:junit4] 2> 664653 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica4 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 664672 T830 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4] 2> 664671 T859 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica3 state=down
[junit4:junit4] 2> 664670 T820 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica4 state=down
[junit4:junit4] 2> 664667 T858 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica3 state=down
[junit4:junit4] 2> 664667 T803 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=down
[junit4:junit4] 2> 664666 T801 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4] 2> 664676 T816 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica4 state=down
[junit4:junit4] 2> 664675 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica5 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 664684 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 664687 T844 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica5 state=down
[junit4:junit4] 2> 664689 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 664691 T831 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica1 state=down
[junit4:junit4] 2> 664695 T798 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica2 state=down
[junit4:junit4] 2> 664695 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 664705 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica4 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 664708 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica5 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 664709 T860 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica3 state=down
[junit4:junit4] 2> 664713 T818 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica4 state=down
[junit4:junit4] 2> 664716 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica1 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 664719 T845 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica5 state=down
[junit4:junit4] 2> 664722 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica2 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 664724 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica3 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 664730 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica4 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 664730 T832 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica1 state=down
[junit4:junit4] 2> 664741 T861 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica3 state=down
[junit4:junit4] 2> 664733 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica5 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 664730 T800 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica2 state=down
[junit4:junit4] 2> 664745 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica1 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:49402_hm_dlx%2Fz
[junit4:junit4] 2> 664750 T817 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica4 state=down
[junit4:junit4] 2> 664750 T846 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica5 state=down
[junit4:junit4] 2> 664757 T833 oasc.ZkController.publish publishing core=awholynewcollection_0_shard5_replica1 state=down
[junit4:junit4] 2> 664753 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica2 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:49395_hm_dlx%2Fz
[junit4:junit4] 2> 664760 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica3 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:49408_hm_dlx%2Fz
[junit4:junit4] 2> 664764 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica4 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:49399_hm_dlx%2Fz
[junit4:junit4] 2> 664765 T802 oasc.ZkController.publish publishing core=awholynewcollection_0_shard5_replica2 state=down
[junit4:junit4] 2> 664767 T862 oasc.ZkController.publish publishing core=awholynewcollection_0_shard5_replica3 state=down
[junit4:junit4] 2> 664769 T809 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica5 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:49405_hm_dlx%2Fz
[junit4:junit4] 2> 664773 T815 oasc.ZkController.publish publishing core=awholynewcollection_0_shard5_replica4 state=down
[junit4:junit4] 2> 664777 T847 oasc.ZkController.publish publishing core=awholynewcollection_0_shard5_replica5 state=down
[junit4:junit4] 2> 665860 T808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 665862 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49399_hm_dlx%2Fz_nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 665878 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49408_hm_dlx%2Fz_nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 665893 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49405_hm_dlx%2Fz_nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 665904 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49395_hm_dlx%2Fz_nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 665916 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49402_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49402/hm_dlx/z"}
[junit4:junit4] 2> 665916 T808 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 5
[junit4:junit4] 2> 665936 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 665948 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49402_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49402/hm_dlx/z"}
[junit4:junit4] 2> 665961 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica4",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 665973 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica5",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 665989 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica4",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 666005 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49402_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49402/hm_dlx/z"}
[junit4:junit4] 2> 666021 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 666034 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica5",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 666047 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 666058 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 666078 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica4",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 666091 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 666104 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 666115 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica5",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 666131 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard4",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard4_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49402_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49402/hm_dlx/z"}
[junit4:junit4] 2> 666147 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard4",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard4_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 666173 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard4",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard4_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 666185 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard4",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard4_replica4",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 666200 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard4",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard4_replica5",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 666214 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard5",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard5_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49402_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49402/hm_dlx/z"}
[junit4:junit4] 2> 666226 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard5",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard5_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49408_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49408/hm_dlx/z"}
[junit4:junit4] 2> 666242 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard5",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard5_replica4",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49399_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49399/hm_dlx/z"}
[junit4:junit4] 2> 666253 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard5",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard5_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49395_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49395/hm_dlx/z"}
[junit4:junit4] 2> 666271 T808 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard5",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard5_replica5",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49405_hm_dlx%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49405/hm_dlx/z"}
[junit4:junit4] 2> 666284 T839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T884 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666285 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666305 T807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 666503 T863 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49399_hm_dlx%2Fz_nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 666506 T863 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49399_hm_dlx%252Fz_nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:49399_hm_dlx%252Fz&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2006
[junit4:junit4] 2> 666512 T848 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49395_hm_dlx%2Fz_nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 666512 T848 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49395_hm_dlx%252Fz_nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:49395_hm_dlx%252Fz&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 666684 T835 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 666685 T835 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666687 T835 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 666687 T835 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 666691 T835 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 666691 T835 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 666701 T859 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 666702 T859 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666701 T830 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 666703 T830 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666703 T849 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica5' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard1_replica5
[junit4:junit4] 2> 666705 T849 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666705 T859 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666706 T859 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666702 T820 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard1_replica4
[junit4:junit4] 2> 666707 T820 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666707 T830 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666709 T830 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666706 T835 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666711 T849 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666712 T849 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666717 T820 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666718 T820 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666717 T859 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard2_replica3/'
[junit4:junit4] 2> 666728 T816 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard2_replica4
[junit4:junit4] 2> 666729 T816 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666731 T830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 2> 666731 T831 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard3_replica1
[junit4:junit4] 2> 666742 T831 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666754 T818 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard3_replica4
[junit4:junit4] 2> 666754 T818 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666763 T832 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard4_replica1
[junit4:junit4] 2> 666764 T832 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666765 T860 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard3_replica3
[junit4:junit4] 2> 666769 T860 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666767 T861 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard4_replica3
[junit4:junit4] 2> 666771 T861 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666772 T798 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard3_replica2
[junit4:junit4] 2> 666772 T798 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666769 T800 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard4_replica2
[junit4:junit4] 2> 666773 T800 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666768 T835 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard2_replica1/'
[junit4:junit4] 2> 666782 T846 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica5' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard4_replica5
[junit4:junit4] 2> 666783 T846 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666783 T845 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica5' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard3_replica5
[junit4:junit4] 2> 666784 T845 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666784 T817 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard4_replica4
[junit4:junit4] 2> 666785 T817 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666786 T802 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard5_replica2
[junit4:junit4] 2> 666786 T802 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666785 T833 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard5_replica1
[junit4:junit4] 2> 666787 T833 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666791 T801 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 666792 T801 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666793 T847 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica5' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard5_replica5
[junit4:junit4] 2> 666793 T847 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666788 T815 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard5_replica4
[junit4:junit4] 2> 666794 T815 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666788 T862 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard5_replica3
[junit4:junit4] 2> 666795 T862 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666796 T803 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 666796 T803 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666798 T858 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 666798 T858 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666799 T844 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica5' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard2_replica5
[junit4:junit4] 2> 666799 T844 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 666820 T816 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666821 T816 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666824 T849 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard1_replica5/'
[junit4:junit4] 2> 666826 T818 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666826 T818 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666827 T815 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666828 T815 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666829 T846 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666829 T846 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666830 T847 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666830 T847 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666833 T817 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666833 T817 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666833 T861 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666835 T861 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666833 T860 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666836 T860 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666835 T832 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666838 T832 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666835 T831 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666839 T831 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666834 T862 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666840 T862 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666834 T844 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666841 T844 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666834 T845 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666843 T845 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666833 T858 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666844 T858 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666844 T833 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666845 T833 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666839 T798 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666847 T798 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666845 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard1_replica4/'
[junit4:junit4] 2> 666859 T846 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard4_replica5/'
[junit4:junit4] 2> 666860 T847 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard5_replica5/'
[junit4:junit4] 2> 666862 T816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard2_replica4/'
[junit4:junit4] 2> 666865 T800 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666866 T800 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666868 T818 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard3_replica4/'
[junit4:junit4] 2> 666869 T815 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard5_replica4/'
[junit4:junit4] 2> 666870 T861 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard4_replica3/'
[junit4:junit4] 2> 666872 T817 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/awholynewcollection_0_shard4_replica4/'
[junit4:junit4] 2> 666874 T802 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666874 T802 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666875 T862 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard5_replica3/'
[junit4:junit4] 2> 666876 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard3_replica3/'
[junit4:junit4] 2> 666879 T858 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371714264161/awholynewcollection_0_shard1_replica3/'
[junit4:junit4] 2> 666880 T844 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard2_replica5/'
[junit4:junit4] 2> 666881 T845 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371714261887/awholynewcollection_0_shard3_replica5/'
[junit4:junit4] 2> 666883 T801 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666884 T801 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666887 T832 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard4_replica1/'
[junit4:junit4] 2> 666888 T803 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 666893 T803 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 666895 T831 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard3_replica1/'
[junit4:junit4] 2> 666898 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371714259723/awholynewcollection_0_shard5_replica1/'
[junit4:junit4] 2> 666902 T798 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard3_replica2/'
[junit4:junit4] 2> 666904 T800 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard4_replica2/'
[junit4:junit4] 2> 666907 T802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard5_replica2/'
[junit4:junit4] 2> 666983 T801 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 2> 666994 T803 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/awholynewcollection_0_shard2_replica2/'
[junit4:junit4] 2> 667816 T847 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 667827 T835 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 667876 T862 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 667884 T830 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 667920 T859 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668016 T860 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668077 T849 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668127 T802 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668401 T816 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668420 T831 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668429 T815 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668478 T818 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668491 T801 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> ASYNC NEW_CORE C329 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@150511ef url=http://127.0.0.1:49399/hm_dlx/z/nodes_used_collection_shard1_replica1 node=127.0.0.1:49399_hm_dlx%2Fz C329_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49399_hm_dlx%2Fz, base_url=http://127.0.0.1:49399/hm_dlx/z}
[junit4:junit4] 2> 668509 T879 C329 P49399 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/ core=nodes_used_collection_shard1_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 668509 T879 C329 P49399 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:49399/hm_dlx/z START replicas=[http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4] 2> 668510 T879 C329 P49399 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 668510 T879 C329 P49399 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 668511 T879 C329 P49399 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 668511 T879 C329 P49399 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 668511 T879 C329 P49399 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 668511 T879 C329 P49399 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2/. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 668512 T879 C329 P49399 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C330 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@43f420ba url=http://127.0.0.1:49395/hm_dlx/z/nodes_used_collection_shard2_replica1 node=127.0.0.1:49395_hm_dlx%2Fz C330_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49395_hm_dlx%2Fz, base_url=http://127.0.0.1:49395/hm_dlx/z}
[junit4:junit4] 2> 668515 T880 C330 P49395 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 668515 T880 C330 P49395 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:49395/hm_dlx/z START replicas=[http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 2> 668522 T880 C330 P49395 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 668525 T880 C330 P49395 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 668525 T880 C330 P49395 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 668525 T880 C330 P49395 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 668526 T880 C330 P49395 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 668526 T880 C330 P49395 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 668526 T880 C330 P49395 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C331 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1ac7e25d url=http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2 node=127.0.0.1:49408_hm_dlx%2Fz C331_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:49408_hm_dlx%2Fz, base_url=http://127.0.0.1:49408/hm_dlx/z, leader=true}
[junit4:junit4] 2> 668558 T863 C331 P49408 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/hm_dlx/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=14
[junit4:junit4] 2> 668561 T909 C331 P49408 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 668616 T846 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668622 T858 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668635 T803 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668651 T861 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> ASYNC NEW_CORE C332 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1ac7e25d url=http://127.0.0.1:49408/hm_dlx/z/nodes_used_collection_shard1_replica2 node=127.0.0.1:49408_hm_dlx%2Fz C332_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:49408_hm_dlx%2Fz, base_url=http://127.0.0.1:49408/hm_dlx/z, leader=true}
[junit4:junit4] 2> 668814 T909 C332 P49408 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6058de27 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17446ad5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 668815 T909 C332 P49408 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 668855 T909 C332 P49408 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6058de27 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17446ad5),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6058de27 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17446ad5),segFN=segments_2,generation=2}
[junit4:junit4] 2> 668857 T909 C332 P49408 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> ASYNC NEW_CORE C333 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@9d5de9 url=http://127.0.0.1:49405/hm_dlx/z/nodes_used_collection_shard2_replica2 node=127.0.0.1:49405_hm_dlx%2Fz C333_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:49405_hm_dlx%2Fz, base_url=http://127.0.0.1:49405/hm_dlx/z, leader=true}
[junit4:junit4] 2> 668869 T911 C333 P49405 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 668889 T909 C332 P49408 oass.SolrIndexSearcher.<init> Opening Searcher@7ffca5b8 realtime
[junit4:junit4] 2> 668890 T909 C332 P49408 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 668891 T909 C332 P49408 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica2] webapp=/hm_dlx/z path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 330
[junit4:junit4] 2> ASYNC NEW_CORE C334 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@150511ef url=http://127.0.0.1:49399/hm_dlx/z/nodes_used_collection_shard1_replica1 node=127.0.0.1:49399_hm_dlx%2Fz C334_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49399_hm_dlx%2Fz, base_url=http://127.0.0.1:49399/hm_dlx/z}
[junit4:junit4] 2> 668894 T879 C334 P49399 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 668894 T879 C334 P49399 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 668909 T833 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668915 T844 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668919 T800 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668920 T909 C332 P49408 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 668920 T909 C332 P49408 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/hm_dlx/z path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 668970 T820 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668981 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 668982 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 668983 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 669006 T863 C332 P49408 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/hm_dlx/z path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=11
[junit4:junit4] 2> 669009 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 669044 T879 C334 P49399 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index.20130620094439081
[junit4:junit4] 2> 669081 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@66cfeead lockFactory=org.apache.lucene.store.NativeFSLockFactory@25f807c4) fullCopy=false
[junit4:junit4] 2> 669095 T911 C333 P49405 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c823d00 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1da3fa09),segFN=segments_1,generation=1}
[junit4:junit4] 2> 669106 T911 C333 P49405 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 669108 T909 C332 P49408 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/hm_dlx/z path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 669115 T832 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 669118 T817 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 669126 T911 C333 P49405 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c823d00 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1da3fa09),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c823d00 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1da3fa09),segFN=segments_2,generation=2}
[junit4:junit4] 2> 669127 T911 C333 P49405 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 669128 T911 C333 P49405 oass.SolrIndexSearcher.<init> Opening Searcher@1bd54242 realtime
[junit4:junit4] 2> 669129 T911 C333 P49405 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 669129 T911 C333 P49405 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp=/hm_dlx/z path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 261
[junit4:junit4] 2> ASYNC NEW_CORE C335 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@43f420ba url=http://127.0.0.1:49395/hm_dlx/z/nodes_used_collection_shard2_replica1 node=127.0.0.1:49395_hm_dlx%2Fz C335_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:49395_hm_dlx%2Fz, base_url=http://127.0.0.1:49395/hm_dlx/z}
[junit4:junit4] 2> 669142 T880 C335 P49395 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 669143 T880 C335 P49395 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 669158 T848 C333 P49405 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/hm_dlx/z path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 669175 T879 C334 P49399 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 669177 T848 C333 P49405 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 669177 T848 C333 P49405 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/hm_dlx/z path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=4
[junit4:junit4] 2> 669179 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 669179 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 669179 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 669203 T914 C333 P49405 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/hm_dlx/z path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=10
[junit4:junit4] 2> 669206 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 669214 T879 C334 P49399 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 669215 T879 C334 P49399 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 669216 T880 C335 P49395 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371714256139/nodes_used_collection_shard2_replica1/data/index.20130620094439278
[junit4:junit4] 2> 669217 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4456d453 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f39ee05) fullCopy=false
[junit4:junit4] 2> 669227 T879 C334 P49399 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516fc4ce lockFactory=org.apache.lucene.store.NativeFSLockFactory@4cc2e9fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516fc4ce lockFactory=org.apache.lucene.store.NativeFSLockFactory@4cc2e9fc),segFN=segments_2,generation=2}
[junit4:junit4] 2> 669227 T879 C334 P49399 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 669228 T879 C334 P49399 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 669229 T879 C334 P49399 oass.SolrIndexSearcher.<init> Opening Searcher@6c4b0d5d main
[junit4:junit4] 2> 669231 T876 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@6c4b0d5d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 669233 T879 C334 P49399 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index.20130620094439081 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index.20130620094439081;done=true>>]
[junit4:junit4] 2> 669233 T879 C334 P49399 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index.20130620094439081
[junit4:junit4] 2> 669241 T845 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 669249 T879 C334 P49399 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371714257544/nodes_used_collection_shard1_replica1/data/index.20130620094439081
[junit4:junit4] 2> 669250 T879 C334 P49399 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 669250 T879 C334 P49399 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 669250 T879 C334 P49399 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4] 2> 669255 T879 C334 P49399 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 669260 T911 C333 P49405 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/hm_dlx/z path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 669278 T880 C335 P49395 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 669282 T880 C335 P49395 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 669282 T880 C335 P49395 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:ju
[...truncated too long message...]
:"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49532_xm_bai%2Fy",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49532/xm_bai/y"},
[junit4:junit4] 1> "5":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49539_xm_bai%2Fy",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49539/xm_bai/y",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:49521_xm_bai%2Fy",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49521/xm_bai/y",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49532_xm_bai%2Fy (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49539_xm_bai%2Fy (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49521_xm_bai%2Fy (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49525_xm_bai%2Fy (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89896679124566026-4-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89896679124566030-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89896679124566028-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49539_xm_bai%2Fy",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49539/xm_bai/y"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89896679124566019-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49521_xm_bai%2Fy",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49521/xm_bai/y"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89896679124566026-127.0.0.1:49532_xm_bai%2Fy-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89896679124566030-127.0.0.1:49525_xm_bai%2Fy-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89896679124566028-127.0.0.1:49539_xm_bai%2Fy-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89896679124566019-127.0.0.1:49521_xm_bai%2Fy-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89896679124566019-127.0.0.1:49521_xm_bai%2Fy-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=C44103AB69D2E3 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=Australia/Lindeman -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 140s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49525/xm_bai/y/collection1lastClient and got 267 from http://127.0.0.1:49532/xm_bai/y/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([C44103AB69D2E3:8122CF1BDC36B2DF]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 994489 T1080 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 140413 T1079 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=SimpleText), _version_=PostingsFormat(name=Asserting), rnd_b=Pulsing41(freqCutoff=2 minBlockSize=50 maxBlockSize=160), intDefault=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=Memory doPackFST= true), id=Pulsing41(freqCutoff=2 minBlockSize=50 maxBlockSize=160), a_t=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=Pulsing41(freqCutoff=2 minBlockSize=50 maxBlockSize=160), range_facet_si=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Memory doPackFST= true), a_si=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=lt, timezone=Australia/Lindeman
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=141757232,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestManagedSchema, TestDocumentBuilder, SystemInfoHandlerTest, PingRequestHandlerTest, TestAddFieldRealTimeGet, UnloadDistributedZkTest, ScriptEngineTest, CopyFieldTest, DOMUtilTest, SolrPluginUtilsTest, TestDefaultSearchFieldResource, TestRealTimeGet, TestFiltering, DateMathParserTest, TestZkChroot, NoCacheHeaderTest, PreAnalyzedFieldTest, TestMaxScoreQueryParser, AliasIntegrationTest, TestCSVLoader, TestGroupingSearch, TestDocSet, URLClassifyProcessorTest, UpdateRequestProcessorFactoryTest, SpellCheckCollatorTest, TestJmxMonitoredMap, TestStressRecovery, DistributedQueryElevationComponentTest, IndexBasedSpellCheckerTest, TestCloudManagedSchema, CollectionsAPIDistributedZkTest, OutputWriterTest, QueryParsingTest, SolrInfoMBeanTest, TestSolrQueryParserResource, TestSolrXMLSerializer, RequestHandlersTest, SyncSliceTest]
[junit4:junit4] Completed in 140.98s, 1 test, 1 failure <<< FAILURES!
[...truncated 805 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 1 failure, 18 ignored (6 assumptions)
Total time: 110 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 578 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/578/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:51233 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51233 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([24ACDDE86AA4AFF2:A54A53F01DFBCFCE]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<159> but was:<25>
Stack Trace:
java.lang.AssertionError: expected:<159> but was:<25>
at __randomizedtesting.SeedInfo.seed([24ACDDE86AA4AFF2:A54A53F01DFBCFCE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9286 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 1098173 T1219 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1098197 T1219 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1371689186273
[junit4:junit4] 2> 1098210 T1219 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1098227 T1220 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1098328 T1219 oasc.ZkTestServer.run start zk server on port:49351
[junit4:junit4] 2> 1098332 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1098801 T1226 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ce9fc96 name:ZooKeeperConnection Watcher:127.0.0.1:49351 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1098801 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1098802 T1219 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1098960 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1099018 T1228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60382f8 name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1099018 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1099020 T1219 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1099035 T1219 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1099052 T1219 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1099084 T1219 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1099097 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1099100 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1099130 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1099132 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1099166 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1099168 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1099202 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1099204 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1099217 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1099219 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1099246 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1099248 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1099325 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1099327 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1099413 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1099415 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1099456 T1219 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1099459 T1219 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1100570 T1219 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1100606 T1219 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49354
[junit4:junit4] 2> 1100607 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1100609 T1219 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1100609 T1219 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562
[junit4:junit4] 2> 1100610 T1219 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/solr.xml
[junit4:junit4] 2> 1100611 T1219 oasc.CoreContainer.<init> New CoreContainer 452975813
[junit4:junit4] 2> 1100613 T1219 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/'
[junit4:junit4] 2> 1100614 T1219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/'
[junit4:junit4] 2> 1100934 T1219 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1100936 T1219 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1100937 T1219 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1100939 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1100942 T1219 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1100949 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1100950 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1100951 T1219 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1100954 T1219 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1100955 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1101000 T1219 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1101001 T1219 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49351/solr
[junit4:junit4] 2> 1101003 T1219 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1101006 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1101016 T1239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14b5f487 name:ZooKeeperConnection Watcher:127.0.0.1:49351 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1101019 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1101034 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1101078 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1101089 T1241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@323639e6 name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1101090 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1101110 T1219 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1101134 T1219 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1101155 T1219 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1101167 T1219 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49354_
[junit4:junit4] 2> 1101175 T1219 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49354_
[junit4:junit4] 2> 1101212 T1219 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1101252 T1219 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1101263 T1219 oasc.Overseer.start Overseer (id=89895022513422339-127.0.0.1:49354_-n_0000000000) starting
[junit4:junit4] 2> 1101294 T1219 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1101320 T1243 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1101329 T1219 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1101341 T1219 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1101350 T1219 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1101380 T1242 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1101388 T1244 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1101389 T1244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1102901 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1102904 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49354_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49354"}
[junit4:junit4] 2> 1102905 T1242 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1102905 T1242 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1102926 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1103398 T1244 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/collection1
[junit4:junit4] 2> 1103398 T1244 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1103401 T1244 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1103401 T1244 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1103405 T1244 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/collection1/'
[junit4:junit4] 2> 1103408 T1244 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1103411 T1244 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/collection1/lib/README' to classloader
[junit4:junit4] 2> 1103580 T1244 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1103773 T1244 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1103780 T1244 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1103797 T1244 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1105793 T1244 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1105837 T1244 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1105848 T1244 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1105934 T1244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1105949 T1244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1105965 T1244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1105979 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1105986 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1105986 T1244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1105992 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1105993 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1105993 T1244 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1105993 T1244 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371689187562/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data/
[junit4:junit4] 2> 1105994 T1244 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177766e
[junit4:junit4] 2> 1105994 T1244 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1105995 T1244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data
[junit4:junit4] 2> 1105996 T1244 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data/index/
[junit4:junit4] 2> 1105996 T1244 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1105997 T1244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data/index
[junit4:junit4] 2> 1106004 T1244 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@48459246 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e5e81a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1106004 T1244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1106009 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1106009 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1106010 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1106013 T1244 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1106014 T1244 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1106014 T1244 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1106015 T1244 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1106017 T1244 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1106018 T1244 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1106047 T1244 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1106077 T1244 oass.SolrIndexSearcher.<init> Opening Searcher@6532a4b1 main
[junit4:junit4] 2> 1106078 T1244 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/control/data/tlog
[junit4:junit4] 2> 1106080 T1244 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1106081 T1244 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1106090 T1245 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6532a4b1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1106094 T1244 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1106095 T1244 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49354 collection:control_collection shard:shard1
[junit4:junit4] 2> 1106099 T1244 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1106137 T1244 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1106167 T1244 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1106168 T1244 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1106168 T1244 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49354/collection1/
[junit4:junit4] 2> 1106169 T1244 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1106169 T1244 oasc.SyncStrategy.syncToMe http://127.0.0.1:49354/collection1/ has no replicas
[junit4:junit4] 2> 1106170 T1244 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49354/collection1/
[junit4:junit4] 2> 1106171 T1244 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1107491 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1107556 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1107590 T1244 oasc.ZkController.register We are http://127.0.0.1:49354/collection1/ and leader is http://127.0.0.1:49354/collection1/
[junit4:junit4] 2> 1107591 T1244 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49354
[junit4:junit4] 2> 1107591 T1244 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1107592 T1244 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1107592 T1244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1107598 T1244 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1107607 T1219 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1107612 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1107613 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1107663 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1107669 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1107681 T1248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59ab08a4 name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1107681 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1107693 T1219 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1107705 T1219 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1108773 T1219 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1108786 T1219 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49358
[junit4:junit4] 2> 1108788 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1108789 T1219 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1108790 T1219 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770
[junit4:junit4] 2> 1108790 T1219 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/solr.xml
[junit4:junit4] 2> 1108792 T1219 oasc.CoreContainer.<init> New CoreContainer 1251248242
[junit4:junit4] 2> 1108794 T1219 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/'
[junit4:junit4] 2> 1108795 T1219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/'
[junit4:junit4] 2> 1109122 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1109125 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49354__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49354_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49354"}
[junit4:junit4] 2> 1109181 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1109182 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1109228 T1219 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1109229 T1219 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1109230 T1219 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1109232 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1109233 T1219 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1109233 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1109234 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1109235 T1219 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1109237 T1219 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1109238 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1109293 T1219 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1109294 T1219 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49351/solr
[junit4:junit4] 2> 1109296 T1219 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1109300 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1109310 T1259 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@716c5161 name:ZooKeeperConnection Watcher:127.0.0.1:49351 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1109311 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1109326 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1109370 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1109397 T1261 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2815ab69 name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1109398 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1109424 T1219 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1110442 T1219 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49358_
[junit4:junit4] 2> 1110447 T1219 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49358_
[junit4:junit4] 2> 1110458 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1110467 T1241 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1110480 T1248 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1110481 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1110539 T1262 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1110540 T1262 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1110542 T1261 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1110714 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1110718 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49358_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49358"}
[junit4:junit4] 2> 1110718 T1242 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 1110719 T1242 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1110737 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1110738 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1110741 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1111549 T1262 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/collection1
[junit4:junit4] 2> 1111552 T1262 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1111555 T1262 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1111556 T1262 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1111566 T1262 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/collection1/'
[junit4:junit4] 2> 1111569 T1262 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1111571 T1262 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/collection1/lib/README' to classloader
[junit4:junit4] 2> 1111716 T1262 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1111895 T1262 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1111901 T1262 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1111914 T1262 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1113777 T1262 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1113821 T1262 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1113830 T1262 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1113925 T1262 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1113937 T1262 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1113952 T1262 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1113958 T1262 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1113960 T1262 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1113961 T1262 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1113967 T1262 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1113968 T1262 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1113969 T1262 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1113970 T1262 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371689195770/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1/
[junit4:junit4] 2> 1113970 T1262 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177766e
[junit4:junit4] 2> 1113974 T1262 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1113977 T1262 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1
[junit4:junit4] 2> 1113978 T1262 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1/index/
[junit4:junit4] 2> 1113978 T1262 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1113981 T1262 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1/index
[junit4:junit4] 2> 1113997 T1262 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d1479c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@37d9cf97),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1113998 T1262 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1114010 T1262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1114011 T1262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1114013 T1262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1114015 T1262 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1114017 T1262 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1114017 T1262 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1114018 T1262 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1114020 T1262 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1114021 T1262 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1114059 T1262 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1114089 T1262 oass.SolrIndexSearcher.<init> Opening Searcher@6d1e3d1c main
[junit4:junit4] 2> 1114090 T1262 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty1/tlog
[junit4:junit4] 2> 1114093 T1262 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1114094 T1262 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1114103 T1263 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d1e3d1c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1114107 T1262 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1114108 T1262 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49358 collection:collection1 shard:shard1
[junit4:junit4] 2> 1114111 T1262 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1114156 T1262 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1114169 T1262 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1114170 T1262 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1114171 T1262 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49358/collection1/
[junit4:junit4] 2> 1114171 T1262 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1114172 T1262 oasc.SyncStrategy.syncToMe http://127.0.0.1:49358/collection1/ has no replicas
[junit4:junit4] 2> 1114172 T1262 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49358/collection1/
[junit4:junit4] 2> 1114172 T1262 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1115352 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1115417 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1115418 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1115434 T1262 oasc.ZkController.register We are http://127.0.0.1:49358/collection1/ and leader is http://127.0.0.1:49358/collection1/
[junit4:junit4] 2> 1115434 T1262 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49358
[junit4:junit4] 2> 1115435 T1262 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1115435 T1262 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1115436 T1262 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1115437 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1115448 T1262 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1115456 T1219 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1115458 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1115462 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1116717 T1219 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1116731 T1219 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49361
[junit4:junit4] 2> 1116733 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1116735 T1219 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1116736 T1219 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600
[junit4:junit4] 2> 1116737 T1219 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/solr.xml
[junit4:junit4] 2> 1116738 T1219 oasc.CoreContainer.<init> New CoreContainer 820801163
[junit4:junit4] 2> 1116740 T1219 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/'
[junit4:junit4] 2> 1116742 T1219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/'
[junit4:junit4] 2> 1117061 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1117070 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49358__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49358_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49358"}
[junit4:junit4] 2> 1117116 T1219 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1117118 T1219 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1117118 T1219 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1117119 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1117120 T1219 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1117125 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1117127 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1117128 T1219 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1117129 T1219 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1117130 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1117189 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1117201 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1117202 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1117221 T1219 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1117223 T1219 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49351/solr
[junit4:junit4] 2> 1117225 T1219 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1117232 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1117240 T1275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6927e837 name:ZooKeeperConnection Watcher:127.0.0.1:49351 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1117242 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1117258 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1117308 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1117321 T1277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4fb8bb1f name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1117322 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1117351 T1219 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1118389 T1219 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49361_
[junit4:junit4] 2> 1118392 T1219 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49361_
[junit4:junit4] 2> 1118405 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1118413 T1277 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1118416 T1248 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1118416 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1118423 T1261 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1118424 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1118427 T1241 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1118466 T1278 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1118467 T1278 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1118784 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1118788 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1118789 T1242 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1118789 T1242 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1118816 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1118817 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1118824 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1118826 T1277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1119474 T1278 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1
[junit4:junit4] 2> 1119474 T1278 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1119476 T1278 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1119477 T1278 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1119482 T1278 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/'
[junit4:junit4] 2> 1119484 T1278 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1119484 T1278 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/lib/README' to classloader
[junit4:junit4] 2> 1119649 T1278 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1119929 T1278 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1119946 T1278 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1119963 T1278 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1121969 T1278 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1122000 T1278 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1122009 T1278 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1122114 T1278 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1122129 T1278 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1122142 T1278 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1122160 T1278 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1122175 T1278 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1122176 T1278 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1122256 T1278 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1122259 T1278 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1122260 T1278 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1122265 T1278 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/
[junit4:junit4] 2> 1122266 T1278 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177766e
[junit4:junit4] 2> 1122267 T1278 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1122269 T1278 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2
[junit4:junit4] 2> 1122270 T1278 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index/
[junit4:junit4] 2> 1122270 T1278 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1122271 T1278 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index
[junit4:junit4] 2> 1122280 T1278 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1122281 T1278 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1122292 T1278 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1122293 T1278 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1122295 T1278 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1122296 T1278 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1122298 T1278 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1122298 T1278 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1122301 T1278 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1122302 T1278 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1122304 T1278 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1122362 T1278 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1122385 T1278 oass.SolrIndexSearcher.<init> Opening Searcher@46d3ac2 main
[junit4:junit4] 2> 1122386 T1278 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/tlog
[junit4:junit4] 2> 1122390 T1278 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1122391 T1278 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1122401 T1279 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46d3ac2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1122408 T1278 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1122409 T1278 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49361 collection:collection1 shard:shard1
[junit4:junit4] 2> 1122422 T1278 oasc.ZkController.register We are http://127.0.0.1:49361/collection1/ and leader is http://127.0.0.1:49358/collection1/
[junit4:junit4] 2> 1122422 T1278 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49361
[junit4:junit4] 2> 1122423 T1278 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1122423 T1278 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C624 name=collection1 org.apache.solr.core.SolrCore@7d3f174f url=http://127.0.0.1:49361/collection1 node=127.0.0.1:49361_ C624_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49361_, base_url=http://127.0.0.1:49361}
[junit4:junit4] 2> 1122425 T1280 C624 P49361 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1122426 T1280 C624 P49361 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1122427 T1280 C624 P49361 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1122427 T1280 C624 P49361 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1122433 T1280 C624 P49361 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1122455 T1278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1122462 T1219 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1122463 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1122464 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1122505 T1219 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1122512 T1254 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49361__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> ASYNC NEW_CORE C625 name=collection1 org.apache.solr.core.SolrCore@60c30e1d url=http://127.0.0.1:49354/collection1 node=127.0.0.1:49354_ C625_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49354_, base_url=http://127.0.0.1:49354, leader=true}
[junit4:junit4] 2> 1122573 T1232 C625 P49354 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@48459246 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e5e81a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1122582 T1232 C625 P49354 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1122591 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438320385745485824)]} 0 6
[junit4:junit4] 2> 1122633 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10001 (1438320385723465728)]} 0 69
[junit4:junit4] 2> ASYNC NEW_CORE C626 name=collection1 org.apache.solr.core.SolrCore@1cb59d78 url=http://127.0.0.1:49358/collection1 node=127.0.0.1:49358_ C626_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49358_, base_url=http://127.0.0.1:49358, leader=true}
[junit4:junit4] 2> 1122675 T1257 C626 P49358 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d1479c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@37d9cf97),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1122677 T1257 C626 P49358 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1122683 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10001 (1438320385832517632)]} 0 14
[junit4:junit4] 2> 1122688 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438320385834614784)]} 0 17
[junit4:junit4] 2> 1122730 T1219 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49361
[junit4:junit4] 2> 1122730 T1219 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=820801163
[junit4:junit4] 2> 1122760 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10002 (1438320385925840896)]} 0 3
[junit4:junit4] 2> 1122764 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438320385930035200)]} 0 2
[junit4:junit4] 2> 1122776 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10002 (1438320385942618112)]} 0 3
[junit4:junit4] 2> 1122781 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438320385947860992)]} 0 3
[junit4:junit4] 2> 1122878 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10003 (1438320386048524288)]} 0 3
[junit4:junit4] 2> 1122881 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438320386051670016)]} 0 3
[junit4:junit4] 2> 1122885 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10003 (1438320386057961472)]} 0 2
[junit4:junit4] 2> 1122897 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438320386070544384)]} 0 2
[junit4:junit4] 2> 1122935 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10004 (1438320386105147392)]} 0 7
[junit4:junit4] 2> 1122940 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438320386115633152)]} 0 2
[junit4:junit4] 2> 1122944 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10004 (1438320386118778880)]} 0 2
[junit4:junit4] 2> 1122951 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438320386126118912)]} 0 3
[junit4:junit4] 2> 1123027 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10004 (-1438320386207907840)]} 0 0
[junit4:junit4] 2> 1123039 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10004 (-1438320386219442176)]} 0 1
[junit4:junit4] 2> 1123053 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10005 (1438320386233073664)]} 0 3
[junit4:junit4] 2> 1123066 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[4 (-1438320386237267968)]} 0 11
[junit4:junit4] 2> 1123073 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10005 (1438320386254045184)]} 0 2
[junit4:junit4] 2> 1123080 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[4 (-1438320386261385216)]} 0 2
[junit4:junit4] 2> 1123101 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1438320386281308160)]} 0 5
[junit4:junit4] 2> 1123111 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1438320386292842496)]} 0 3
[junit4:junit4] 2> 1123114 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10005 (-1438320386299133952)]} 0 1
[junit4:junit4] 2> 1123125 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10005 (-1438320386310668288)]} 0 0
[junit4:junit4] 2> 1123152 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10006 (1438320386326396928)]} 0 13
[junit4:junit4] 2> 1123162 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[5 (-1438320386349465600)]} 0 1
[junit4:junit4] 2> 1123172 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[5 (-1438320386358902784)]} 0 1
[junit4:junit4] 2> 1123207 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1438320386394554368)]} 0 2
[junit4:junit4] 2> 1123209 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10006 (1438320386386165760)]} 0 14
[junit4:junit4] 2> 1123234 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1438320386423914496)]} 0 2
[junit4:junit4] 2> 1123322 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10007 (1438320386515140608)]} 0 3
[junit4:junit4] 2> 1123327 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1438320386521432064)]} 0 1
[junit4:junit4] 2> 1123337 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10007 (1438320386530869248)]} 0 3
[junit4:junit4] 2> 1123352 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1438320386536112128)]} 0 13
[junit4:junit4] 2> 1123361 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10008 (1438320386556035072)]} 0 3
[junit4:junit4] 2> 1123371 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10008 (1438320386566520832)]} 0 2
[junit4:junit4] 2> 1123385 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438320386579103744)]} 0 5
[junit4:junit4] 2> 1123397 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1123404 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49361__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1123410 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1438320386606366720)]} 0 4
[junit4:junit4] 2> 1123441 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10008 (-1438320386640969728)]} 0 1
[junit4:junit4] 2> 1123457 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10008 (-1438320386657746944)]} 0 1
[junit4:junit4] 2> 1123462 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1123462 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1123463 T1277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1123471 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1123485 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[8 (-1438320386688155648)]} 0 1
[junit4:junit4] 2> 1123488 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10009 (1438320386678718464)]} 0 13
[junit4:junit4] 2> 1123515 T1254 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49361__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1123516 T1254 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49361__collection1&state=recovering&nodeName=127.0.0.1:49361_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> C624_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49361_, base_url=http://127.0.0.1:49361}
[junit4:junit4] 2> 1123575 T1273 C624 P49361 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1123577 T1273 C624 P49361 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1123586 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10009 (1438320386722758656)]} 0 20
[junit4:junit4] 2> 1123589 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10009 (1438320386722758656)]} 0 71
[junit4:junit4] 2> 1123595 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[8 (-1438320386701787136)]} 0 10
[junit4:junit4] 2> 1123599 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[8 (-1438320386701787136)]} 0 101
[junit4:junit4] 2> 1123621 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1438320386825519104)]} 0 7
[junit4:junit4] 2> 1123640 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1438320386838102016)]} 0 3
[junit4:junit4] 2> 1123643 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1438320386838102016)]} 0 15
[junit4:junit4] 2> 1123652 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10010 (1438320386862219264)]} 0 2
[junit4:junit4] 2> 1123672 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10010 (1438320386873753600)]} 0 3
[junit4:junit4] 2> 1123674 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10010 (1438320386873753600)]} 0 12
[junit4:junit4] 2> 1123692 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10011 (1438320386903113728)]} 0 2
[junit4:junit4] 2> 1123716 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1438320386928279552)]} 0 2
[junit4:junit4] 2> 1123716 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10011 (1438320386912550912)]} 0 2
[junit4:junit4] 2> 1123722 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10011 (1438320386912550912)]} 0 24
[junit4:junit4] 2> 1123732 T1219 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1123733 T1219 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1123738 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1438320386944008192)]} 0 3
[junit4:junit4] 2> 1123739 T1219 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=127.0.0.1:49361__collection1core=collection1
[junit4:junit4] 2> 1123743 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1438320386944008192)]} 0 14
[junit4:junit4] 2> 1123762 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1438320386976514048)]} 0 2
[junit4:junit4] 2> 1123781 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10012 (1438320386996436992)]} 0 2
[junit4:junit4] 2> 1123792 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1438320386989096960)]} 0 2
[junit4:junit4] 2> 1123794 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1438320386989096960)]} 0 22
[junit4:junit4] 2> 1123796 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10012 (1438320387004825600)]} 0 2
[junit4:junit4] 2> 1123797 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10012 (1438320387004825600)]} 0 11
[junit4:junit4] 2> 1123846 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10009 (-1438320387066691584)]} 0 1
[junit4:junit4] 2> 1123855 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1438320387074031616)]} 0 2
[junit4:junit4] 2> 1123879 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10009 (-1438320387089760256)]} 0 1
[junit4:junit4] 2> 1123886 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1438320387090808832)]} 0 3
[junit4:junit4] 2> 1123889 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1438320387090808832)]} 0 20
[junit4:junit4] 2> 1123891 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10009 (-1438320387089760256)]} 0 24
[junit4:junit4] 2> 1123906 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10013 (1438320387127508992)]} 0 3
[junit4:junit4] 2> 1123921 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[9 (-1438320387145334784)]} 0 0
[junit4:junit4] 2> 1123928 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10013 (1438320387140091904)]} 0 2
[junit4:junit4] 2> 1123931 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10013 (1438320387140091904)]} 0 15
[junit4:junit4] 2> 1123936 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[9 (-1438320387152674816)]} 0 1
[junit4:junit4] 2> 1123940 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[9 (-1438320387152674816)]} 0 12
[junit4:junit4] 2> 1123948 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438320387171549184)]} 0 2
[junit4:junit4] 2> 1123972 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1438320387183083520)]} 0 3
[junit4:junit4] 2> 1123975 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10014 (1438320387199860736)]} 0 2
[junit4:junit4] 2> 1123975 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1438320387183083520)]} 0 19
[junit4:junit4] 2> 1123997 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10014 (1438320387212443648)]} 0 3
[junit4:junit4] 2> 1124002 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10014 (1438320387212443648)]} 0 17
[junit4:junit4] 2> 1124017 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1438320387244949504)]} 0 2
[junit4:junit4] 2> 1124022 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10015 (1438320387249143808)]} 0 2
[junit4:junit4] 2> 1124053 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1438320387255435264)]} 0 2
[junit4:junit4] 2> 1124056 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1438320387255435264)]} 0 31
[junit4:junit4] 2> 1124056 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10015 (1438320387263823872)]} 0 2
[junit4:junit4] 2> 1124064 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10015 (1438320387263823872)]} 0 31
[junit4:junit4] 2> 1124074 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1438320387301572608)]} 0 4
[junit4:junit4] 2> 1124094 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10016 (1438320387324641280)]} 0 2
[junit4:junit4] 2> 1124104 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1438320387322544128)]} 0 2
[junit4:junit4] 2> 1124113 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10016 (1438320387333029888)]} 0 2
[junit4:junit4] 2> 1124114 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1438320387322544128)]} 0 24
[junit4:junit4] 2> 1124115 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10016 (1438320387333029888)]} 0 16
[junit4:junit4] 2> 1124159 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1438320387391750144)]} 0 5
[junit4:junit4] 2> 1124161 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10010 (-1438320387395944448)]} 0 1
[junit4:junit4] 2> 1124173 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10010 (-1438320387405381632)]} 0 0
[junit4:junit4] 2> 1124176 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1438320387404333056)]} 0 3
[junit4:junit4] 2> 1124177 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10010 (-1438320387405381632)]} 0 9
[junit4:junit4] 2> 1124180 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1438320387404333056)]} 0 13
[junit4:junit4] 2> 1124185 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10017 (1438320387421110272)]} 0 2
[junit4:junit4] 2> 1124198 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10017 (1438320387428450304)]} 0 2
[junit4:junit4] 2> 1124200 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10017 (1438320387428450304)]} 0 9
[junit4:junit4] 2> 1124218 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10 (-1438320387456761856)]} 0 1
[junit4:junit4] 2> 1124235 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10 (-1438320387464101888)]} 0 3
[junit4:junit4] 2> 1124239 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10 (-1438320387464101888)]} 0 14
[junit4:junit4] 2> 1124246 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1438320387485073408)]} 0 1
[junit4:junit4] 2> 1124262 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1438320387495559168)]} 0 2
[junit4:junit4] 2> 1124265 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1438320387495559168)]} 0 11
[junit4:junit4] 2> 1124301 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10014 (-1438320387543793664)]} 0 1
[junit4:junit4] 2> 1124315 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10014 (-1438320387551133696)]} 0 1
[junit4:junit4] 2> 1124318 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10014 (-1438320387551133696)]} 0 10
[junit4:junit4] 2> 1124324 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10018 (1438320387566862336)]} 0 2
[junit4:junit4] 2> 1124337 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10018 (1438320387574202368)]} 0 2
[junit4:junit4] 2> 1124340 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10018 (1438320387574202368)]} 0 10
[junit4:junit4] 2> 1124363 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[14 (-1438320387608805376)]} 0 1
[junit4:junit4] 2> 1124380 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[14 (-1438320387616145408)]} 0 0
[junit4:junit4] 2> 1124386 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[14 (-1438320387616145408)]} 0 16
[junit4:junit4] 2> 1124393 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1438320387639214080)]} 0 2
[junit4:junit4] 2> 1124421 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438320387658088448)]} 0 3
[junit4:junit4] 2> 1124424 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1438320387658088448)]} 0 15
[junit4:junit4] 2> 1124427 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10015 (-1438320387675914240)]} 0 1
[junit4:junit4] 2> 1124454 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10015 (-1438320387700031488)]} 0 0
[junit4:junit4] 2> 1124460 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10015 (-1438320387700031488)]} 0 10
[junit4:junit4] 2> 1124468 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10019 (1438320387716808704)]} 0 2
[junit4:junit4] 2> 1124494 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10019 (1438320387738828800)]} 0 2
[junit4:junit4] 2> 1124498 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10019 (1438320387738828800)]} 0 12
[junit4:junit4] 2> 1124501 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[15 (-1438320387753508864)]} 0 1
[junit4:junit4] 2> 1124513 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[15 (-1438320387760848896)]} 0 0
[junit4:junit4] 2> 1124516 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[15 (-1438320387760848896)]} 0 8
[junit4:junit4] 2> 1124528 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1438320387779723264)]} 0 2
[junit4:junit4] 2> 1124549 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438320387789160448)]} 0 2
[junit4:junit4] 2> 1124552 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1438320387789160448)]} 0 17
[junit4:junit4] 2> 1124596 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10017 (-1438320387853123584)]} 0 0
[junit4:junit4] 2> 1124608 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10017 (-1438320387861512192)]} 0 0
[junit4:junit4] 2> 1124611 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10017 (-1438320387861512192)]} 0 7
[junit4:junit4] 2> 1124618 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10020 (1438320387875143680)]} 0 2
[junit4:junit4] 2> 1124631 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10020 (1438320387882483712)]} 0 2
[junit4:junit4] 2> 1124634 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10020 (1438320387882483712)]} 0 11
[junit4:junit4] 2> 1124658 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[17 (-1438320387918135296)]} 0 0
[junit4:junit4] 2> 1124659 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10018 (-1438320387919183872)]} 0 1
[junit4:junit4] 2> 1124671 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10018 (-1438320387925475329)]} 0 0
[junit4:junit4] 2> 1124671 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[17 (-1438320387925475328)]} 0 0
[junit4:junit4] 2> 1124674 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10018 (-1438320387925475329)]} 0 9
[junit4:junit4] 2> 1124674 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[17 (-1438320387925475328)]} 0 10
[junit4:junit4] 2> 1124683 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1438320387943301120)]} 0 2
[junit4:junit4] 2> 1124687 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10021 (1438320387947495424)]} 0 2
[junit4:junit4] 2> 1124703 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438320387955884032)]} 0 4
[junit4:junit4] 2> 1124706 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1438320387955884032)]} 0 13
[junit4:junit4] 2> 1124711 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10021 (1438320387962175488)]} 0 2
[junit4:junit4] 2> 1124714 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10021 (1438320387962175488)]} 0 14
[junit4:junit4] 2> 1124725 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[18 (-1438320387988389888)]} 0 0
[junit4:junit4] 2> 1124743 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[18 (-1438320387997827072)]} 0 1
[junit4:junit4] 2> 1124746 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[18 (-1438320387997827072)]} 0 13
[junit4:junit4] 2> 1124753 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10022 (1438320388007264256)]} 0 11
[junit4:junit4] 2> 1124758 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1438320388020895744)]} 0 2
[junit4:junit4] 2> 1124772 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10022 (1438320388029284352)]} 0 2
[junit4:junit4] 2> 1124774 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10022 (1438320388029284352)]} 0 10
[junit4:junit4] 2> 1124785 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438320388042915840)]} 0 2
[junit4:junit4] 2> 1124787 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1438320388042915840)]} 0 11
[junit4:junit4] 2> 1124813 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1438320388078567424)]} 0 2
[junit4:junit4] 2> 1124827 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438320388085907456)]} 0 1
[junit4:junit4] 2> 1124830 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1438320388085907456)]} 0 12
[junit4:junit4] 2> 1124864 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10020 (-1438320388128899072)]} 0 6
[junit4:junit4] 2> 1124878 T1273 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10020 (-1438320388143579136)]} 0 1
[junit4:junit4] 2> 1124882 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10020 (-1438320388143579136)]} 0 9
[junit4:junit4] 2> 1124890 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10023 (1438320388159307776)]} 0 2
[junit4:junit4] 2> 1124915 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[20 (-1438320388187619328)]} 0 0
[junit4:junit4] 2> 1124922 T1271 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10023 (1438320388173987840)]} 0 2
[junit4:junit4] 2> 1124934 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10023 (1438320388173987840)]} 0 32
[junit4:junit4] 2> 1124944 T1270 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[20 (-1438320388210688000)]} 0 0
[junit4:junit4] 2> 1124947 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[20 (-1438320388210688000)]} 0 20
[junit4:junit4] 2> 1124956 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438320388228513792)]} 0 2
[junit4:junit4] 2> 1124969 T1268 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438320388236902400)]} 0 2
[junit4:junit4] 2> 1124972 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1438320388236902400)]} 0 11
[junit4:junit4] 2> 1125005 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10024 (1438320388280942592)]} 0 2
[junit4:junit4] 2> 1125020 T1272 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10024 (1438320388289331200)]} 0 2
[junit4:junit4] 2> 1125023 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10024 (1438320388289331200)]} 0 11
[junit4:junit4] 2> 1125042 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1438320388316594176)]} 0 4
[junit4:junit4] 2> 1125052 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1125057 T1269 C624 P49361 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49358/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438320388328128512)]} 0 2
[junit4:junit4] 2> 1125065 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1438320388328128512)]} 0 17
[junit4:junit4] 2> 1125072 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49361__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1125095 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1125102 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1125103 T1277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1125105 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1125143 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10021 (-1438320388426694656)]} 0 0
[junit4:junit4] 2> 1125153 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10021 (-1438320388437180416)]} 0 0
[junit4:junit4] 2> 1125158 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[21 (-1438320388442423296)]} 0 1
[junit4:junit4] 2> 1125172 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10025 (1438320388455006208)]} 0 3
[junit4:junit4] 2> 1125175 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[21 (-1438320388460249088)]} 0 0
[junit4:junit4] 2> 1125183 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1438320388466540544)]} 0 2
[junit4:junit4] 2> 1125185 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10025 (1438320388469686272)]} 0 2
[junit4:junit4] 2> 1125190 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1438320388474929152)]} 0 2
[junit4:junit4] 2> 1125289 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10022 (-1438320388579786752)]} 0 1
[junit4:junit4] 2> 1125296 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[22 (-1438320388587126784)]} 0 1
[junit4:junit4] 2> 1125299 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10022 (-1438320388590272512)]} 0 1
[junit4:junit4] 2> 1125304 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[22 (-1438320388595515392)]} 0 0
[junit4:junit4] 2> 1125320 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10026 (1438320388609146880)]} 0 5
[junit4:junit4] 2> 1125322 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438320388613341184)]} 0 2
[junit4:junit4] 2> 1125329 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10026 (1438320388619632640)]} 0 2
[junit4:junit4] 2> 1125331 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1438320388621729792)]} 0 2
[junit4:junit4] 2> 1125348 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438320388639555584)]} 0 2
[junit4:junit4] 2> 1125356 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10027 (1438320388644798464)]} 0 6
[junit4:junit4] 2> 1125363 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1438320388656332800)]} 0 2
[junit4:junit4] 2> 1125368 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10027 (1438320388661575680)]} 0 2
[junit4:junit4] 2> 1125381 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[23 (-1438320388676255744)]} 0 1
[junit4:junit4] 2> 1125390 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[23 (-1438320388685692928)]} 0 1
[junit4:junit4] 2> 1125393 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10023 (-1438320388688838656)]} 0 0
[junit4:junit4] 2> 1125400 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10023 (-1438320388695130112)]} 0 1
[junit4:junit4] 2> 1125405 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1438320388699324416)]} 0 2
[junit4:junit4] 2> 1125410 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10028 (1438320388705615872)]} 0 2
[junit4:junit4] 2> 1125415 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1438320388710858752)]} 0 2
[junit4:junit4] 2> 1125430 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10028 (1438320388725538816)]} 0 3
[junit4:junit4] 2> 1125516 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438320388815716352)]} 0 2
[junit4:junit4] 2> 1125526 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10029 (1438320388824104960)]} 0 4
[junit4:junit4] 2> 1125528 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1438320388829347840)]} 0 2
[junit4:junit4] 2> C624_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49361_, base_url=http://127.0.0.1:49361}
[junit4:junit4] 2> 1125530 T1280 C624 P49361 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49358/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1125531 T1280 C624 P49361 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49361 START replicas=[http://127.0.0.1:49358/collection1/] nUpdates=100
[junit4:junit4] 2> 1125533 T1280 C624 P49361 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1125534 T1280 C624 P49361 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1125534 T1280 C624 P49361 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1125534 T1280 C624 P49361 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1125535 T1280 C624 P49361 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/tlog/tlog.0000000000000000000 refcount=1}}
[junit4:junit4] 2> 1125535 T1280 C624 P49361 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49358/collection1/. core=collection1
[junit4:junit4] 2> 1125536 T1280 C624 P49361 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1125576 T1254 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 1125582 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10029 (1438320388885970944)]} 0 2
[junit4:junit4] 2> 1125607 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438320388911136768)]} 0 3
[junit4:junit4] 2> 1125611 T1252 C626 P49358 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1125626 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1438320388921622528)]} 0 11
[junit4:junit4] 2> 1125648 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10030 (1438320388954128384)]} 0 2
[junit4:junit4] 2> 1125662 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10030 (1438320388966711296)]} 0 4
[junit4:junit4] 2> 1125695 T1252 C626 P49358 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d1479c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@37d9cf97),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d1479c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@37d9cf97),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1125696 T1252 C626 P49358 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1125705 T1232 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[30 (-1438320389014945792)]} 0 0
[junit4:junit4] 2> 1125726 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[30 (-1438320389038014464)]} 0 1
[junit4:junit4] 2> 1125735 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1438320389045354496)]} 0 2
[junit4:junit4] 2> 1125737 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10030 (-1438320389049548800)]} 0 1
[junit4:junit4] 2> 1125757 T1255 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1438320389063180288)]} 0 8
[junit4:junit4] 2> 1125762 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10030 (-1438320389075763200)]} 0 1
[junit4:junit4] 2> 1125771 T1237 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10031 (1438320389083103232)]} 0 3
[junit4:junit4] 2> 1125781 T1257 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10031 (1438320389093588992)]} 0 3
[junit4:junit4] 2> 1125790 T1233 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1438320389103026176)]} 0 2
[junit4:junit4] 2> 1125802 T1252 C626 P49358 oass.SolrIndexSearcher.<init> Opening Searcher@15e5f85 realtime
[junit4:junit4] 2> 1125803 T1252 C626 P49358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1125804 T1252 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 193
[junit4:junit4] 2> 1125811 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1438320389125046272)]} 0 3
[junit4:junit4] 2> 1125819 T1235 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10032 (1438320389133434880)]} 0 2
[junit4:junit4] 2> 1125825 T1280 C624 P49361 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1125826 T1280 C624 P49361 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1125833 T1254 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10032 (1438320389147066368)]} 0 3
[junit4:junit4] 2> 1125851 T1253 C626 P49358 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1125852 T1253 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=5
[junit4:junit4] 2> 1125854 T1280 C624 P49361 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1125855 T1280 C624 P49361 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1125855 T1280 C624 P49361 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1125862 T1257 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1125865 T1280 C624 P49361 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 5
[junit4:junit4] 2> 1125869 T1234 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1438320389183766528)]} 0 4
[junit4:junit4] 2> 1125869 T1280 C624 P49361 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index.20130620034653929
[junit4:junit4] 2> 1125871 T1280 C624 P49361 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@21dae07b lockFactory=org.apache.lucene.store.NativeFSLockFactory@72dca943) fullCopy=false
[junit4:junit4] 2> 1125875 T1252 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1125877 T1256 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1438320389194252288)]} 0 2
[junit4:junit4] 2> 1125885 T1255 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1125896 T1236 C625 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10033 (1438320389214175232)]} 0 2
[junit4:junit4] 2> 1125901 T1254 C626 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 1125905 T1253 C626 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10033 (1438320389223612416)]} 0 3
[junit4:junit4] 2> ASYNC NEW_CORE C627 name=collection1 org.apache.solr.core.SolrCore@60c30e1d url=http://127.0.0.1:49354/collection1 node=127.0.0.1:49354_ C627_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49354_, base_url=http://127.0.0.1:49354, leader=true}
[junit4:junit4] 2> 1126037 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[33 (-1438320389364121600)]} 0 0
[junit4:junit4] 2> ASYNC NEW_CORE C628 name=collection1 org.apache.solr.core.SolrCore@1cb59d78 url=http://127.0.0.1:49358/collection1 node=127.0.0.1:49358_ C628_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49358_, base_url=http://127.0.0.1:49358, leader=true}
[junit4:junit4] 2> 1126055 T1252 C628 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0_1.del&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1126059 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[33 (-1438320389387190272)]} 0 1
[junit4:junit4] 2> 1126067 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1438320389393481728)]} 0 2
[junit4:junit4] 2> 1126071 T1255 C628 P49358 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=_0.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1126095 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1438320389413404672)]} 0 12
[junit4:junit4] 2> ASYNC NEW_CORE C629 name=collection1 org.apache.solr.core.SolrCore@7d3f174f url=http://127.0.0.1:49361/collection1 node=127.0.0.1:49361_ C629_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49361_, base_url=http://127.0.0.1:49361}
[junit4:junit4] 2> 1126095 T1280 C629 P49361 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1126118 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10033 (-1438320389449056256)]} 0 1
[junit4:junit4] 2> 1126118 T1280 C629 P49361 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1126121 T1280 C629 P49361 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1126122 T1280 C629 P49361 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4] 2> 1126125 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10033 (-1438320389456396288)]} 0 1
[junit4:junit4] 2> 1126137 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10034 (1438320389465833472)]} 0 3
[junit4:junit4] 2> 1126147 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10034 (1438320389477367808)]} 0 2
[junit4:junit4] 2> 1126155 T1280 C629 P49361 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1126156 T1280 C629 P49361 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1126178 T1280 C629 P49361 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1126178 T1280 C629 P49361 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1126179 T1280 C629 P49361 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1126192 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1438320389515116544)]} 0 11
[junit4:junit4] 2> 1126205 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1438320389538185216)]} 0 3
[junit4:junit4] 2> 1126239 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10035 (1438320389574885376)]} 0 2
[junit4:junit4] 2> 1126266 T1280 C629 P49361 oass.SolrIndexSearcher.<init> Opening Searcher@f735701 main
[junit4:junit4] 2> 1126268 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10035 (1438320389583273984)]} 0 23
[junit4:junit4] 2> 1126277 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1438320389612634112)]} 0 4
[junit4:junit4] 2> 1126280 T1279 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f735701 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c32/14)}
[junit4:junit4] 2> 1126288 T1280 C629 P49361 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index.20130620034653929 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index.20130620034653929;done=true>>]
[junit4:junit4] 2> 1126293 T1280 C629 P49361 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index.20130620034653929
[junit4:junit4] 2> 1126294 T1280 C629 P49361 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index.20130620034653929
[junit4:junit4] 2> 1126295 T1288 C629 P49361 oasu.UpdateLog$LogReplayer.doReplay WARN Starting log replay tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/tlog/tlog.0000000000000000000 refcount=2} active=true starting pos=3657
[junit4:junit4] 2> 1126296 T1288 C629 P49361 oasu.DirectUpdateHandler2.commit start commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1126297 T1280 C629 P49361 oasc.RecoveryStrategy.replay Replaying buffered documents. core=collection1
[junit4:junit4] 2> 1126293 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1438320389630459904)]} 0 2
[junit4:junit4] 2> 1126397 T1288 C629 P49361 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@98e1fc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4504637a),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1126398 T1288 C629 P49361 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1126417 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10036 (1438320389753143296)]} 0 9
[junit4:junit4] 2> 1126421 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1438320389762580480)]} 0 5
[junit4:junit4] 2> 1126434 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10036 (1438320389775163392)]} 0 5
[junit4:junit4] 2> 1126447 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1438320389791940608)]} 0 3
[junit4:junit4] 2> 1126449 T1288 C629 P49361 oass.SolrIndexSearcher.<init> Opening Searcher@6a5d1f2f main
[junit4:junit4] 2> 1126450 T1288 C629 P49361 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1126452 T1279 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a5d1f2f main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c32/14)}
[junit4:junit4] 2> 1126453 T1288 C629 P49361 oasup.LogUpdateProcessor.finish [collection1] {} 0 158
[junit4:junit4] 2> 1126453 T1288 C629 P49361 oasu.UpdateLog$LogReplayer.run WARN Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=0 positionOfStart=3657}
[junit4:junit4] 2> 1126454 T1280 C629 P49361 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1126455 T1280 C629 P49361 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1126455 T1280 C629 P49361 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1126463 T1280 C629 P49361 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1126465 T1219 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7d3f174f
[junit4:junit4] 2> 1126470 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10037 (1438320389816057856)]} 0 2
[junit4:junit4] 2> 1126488 T1219 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=32,cumulative_deletesById=15,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 1126492 T1219 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1126493 T1219 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1126494 T1219 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1126499 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10037 (1438320389846466560)]} 0 3
[junit4:junit4] 2> 1126505 T1219 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1126506 T1219 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1126507 T1219 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2;done=false>>]
[junit4:junit4] 2> 1126508 T1219 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2
[junit4:junit4] 2> 1126509 T1219 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index;done=false>>]
[junit4:junit4] 2> 1126509 T1219 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371689186271/jetty2/index
[junit4:junit4] 2> 1126517 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1126517 T1277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1126525 T1277 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1126526 T1277 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1126529 T1261 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1126530 T1248 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1126530 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1126530 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1126537 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1438320389886312448)]} 0 3
[junit4:junit4] 2> 1126542 T1241 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1126548 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1438320389898895360)]} 0 2
[junit4:junit4] 2> 1126553 T1219 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 1126563 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1438320389914624000)]} 0 2
[junit4:junit4] 2> 1126572 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1438320389923012608)]} 0 3
[junit4:junit4] 2> 1126582 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10038 (1438320389933498368)]} 0 2
[junit4:junit4] 2> 1126596 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10038 (1438320389949227008)]} 0 2
[junit4:junit4] 2> 1126615 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10039 (1438320389968101376)]} 0 2
[junit4:junit4] 2> 1126638 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1438320389992218624)]} 0 2
[junit4:junit4] 2> 1126639 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10039 (1438320389991170048)]} 0 5
[junit4:junit4] 2> 1126654 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1438320390008995840)]} 0 3
[junit4:junit4] 2> 1126658 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1126661 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49361__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1126689 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1126694 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1126695 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1126713 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[39 (-1438320390072958976)]} 0 0
[junit4:junit4] 2> 1126717 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10040 (1438320390061424640)]} 0 16
[junit4:junit4] 2> 1126722 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[39 (-1438320390082396160)]} 0 0
[junit4:junit4] 2> 1126728 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10040 (1438320390086590464)]} 0 2
[junit4:junit4] 2> 1126732 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1438320390089736192)]} 0 4
[junit4:junit4] 2> 1126743 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1438320390102319104)]} 0 3
[junit4:junit4] 2> 1126768 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10039 (-1438320390129582080)]} 0 1
[junit4:junit4] 2> 1126774 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10039 (-1438320390136922112)]} 0 0
[junit4:junit4] 2> 1126789 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10041 (1438320390150553600)]} 0 2
[junit4:junit4] 2> 1126798 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10041 (1438320390159990784)]} 0 2
[junit4:junit4] 2> 1126804 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1438320390166282240)]} 0 2
[junit4:junit4] 2> 1126814 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1438320390175719424)]} 0 4
[junit4:junit4] 2> 1126816 T1219 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1126839 T1219 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49361
[junit4:junit4] 2> 1126841 T1219 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1126842 T1219 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1126844 T1219 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600
[junit4:junit4] 2> 1126845 T1219 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/solr.xml
[junit4:junit4] 2> 1126846 T1219 oasc.CoreContainer.<init> New CoreContainer 385122335
[junit4:junit4] 2> 1126848 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1438320390212419584)]} 0 2
[junit4:junit4] 2> 1126848 T1219 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/'
[junit4:junit4] 2> 1126851 T1219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/'
[junit4:junit4] 2> 1126907 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10042 (1438320390274285568)]} 0 2
[junit4:junit4] 2> 1126925 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1438320390288965632)]} 0 11
[junit4:junit4] 2> 1126931 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10042 (1438320390294208512)]} 0 7
[junit4:junit4] 2> 1126969 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438320390339297280)]} 0 2
[junit4:junit4] 2> 1126979 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10043 (1438320390349783040)]} 0 2
[junit4:junit4] 2> 1126989 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1438320390350831616)]} 0 12
[junit4:junit4] 2> 1127026 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10043 (1438320390366560256)]} 0 33
[junit4:junit4] 2> 1127031 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[42 (-1438320390406406144)]} 0 1
[junit4:junit4] 2> 1127040 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[42 (-1438320390413746176)]} 0 2
[junit4:junit4] 2> 1127075 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1438320390450446336)]} 0 2
[junit4:junit4] 2> 1127079 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10044 (1438320390454640640)]} 0 2
[junit4:junit4] 2> 1127093 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1438320390458834944)]} 0 12
[junit4:junit4] 2> 1127097 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10044 (1438320390465126400)]} 0 11
[junit4:junit4] 2> 1127140 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10042 (-1438320390519652352)]} 0 1
[junit4:junit4] 2> 1127148 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[44 (-1438320390528040960)]} 0 1
[junit4:junit4] 2> 1127161 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10042 (-1438320390541672448)]} 0 1
[junit4:junit4] 2> 1127164 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[44 (-1438320390545866752)]} 0 1
[junit4:junit4] 2> 1127174 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1438320390554255360)]} 0 3
[junit4:junit4] 2> 1127182 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1438320390563692544)]} 0 2
[junit4:junit4] 2> 1127191 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10045 (1438320390573129728)]} 0 2
[junit4:junit4] 2> 1127210 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10045 (1438320390592004096)]} 0 3
[junit4:junit4] 2> 1127268 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10044 (-1438320390653870080)]} 0 1
[junit4:junit4] 2> 1127281 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10044 (-1438320390668550144)]} 0 0
[junit4:junit4] 2> 1127285 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1438320390670647296)]} 0 2
[junit4:junit4] 2> 1127301 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1438320390687424512)]} 0 2
[junit4:junit4] 2> 1127306 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10046 (1438320390692667392)]} 0 3
[junit4:junit4] 2> 1127315 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10046 (1438320390702104576)]} 0 2
[junit4:junit4] 2> 1127334 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1438320390722027520)]} 0 2
[junit4:junit4] 2> 1127356 T1219 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1127358 T1219 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1127359 T1219 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1127360 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1127362 T1219 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1127363 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1127364 T1219 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1127365 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1438320390755581952)]} 0 2
[junit4:junit4] 2> 1127369 T1219 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1127380 T1219 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1127381 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1127408 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10047 (1438320390800670720)]} 0 2
[junit4:junit4] 2> 1127415 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438320390805913600)]} 0 4
[junit4:junit4] 2> 1127430 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10047 (1438320390819545088)]} 0 6
[junit4:junit4] 2> 1127433 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1438320390823739392)]} 0 5
[junit4:junit4] 2> 1127441 T1219 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1127444 T1219 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49351/solr
[junit4:junit4] 2> 1127445 T1219 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1127452 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10048 (1438320390845759488)]} 0 3
[junit4:junit4] 2> 1127464 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10048 (1438320390858342400)]} 0 3
[junit4:junit4] 2> 1127466 T1299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@359b0b7e name:ZooKeeperConnection Watcher:127.0.0.1:49351 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1127466 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1127468 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1127482 T1219 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1127523 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10049 (1438320390921256960)]} 0 2
[junit4:junit4] 2> 1127532 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[49 (-1438320390930694144)]} 0 0
[junit4:junit4] 2> 1127538 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10049 (1438320390936985600)]} 0 2
[junit4:junit4] 2> 1127544 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[49 (-1438320390944325632)]} 0 1
[junit4:junit4] 2> 1127550 T1219 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1127558 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438320390957957120)]} 0 2
[junit4:junit4] 2> 1127562 T1301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48b75e7f name:ZooKeeperConnection Watcher:127.0.0.1:49351/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1127567 T1219 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1127568 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1438320390966345728)]} 0 3
[junit4:junit4] 2> 1127599 T1219 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1127618 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1438320391019823104)]} 0 3
[junit4:junit4] 2> 1127621 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10049 (-1438320391025065984)]} 0 0
[junit4:junit4] 2> 1127629 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1438320391029260288)]} 0 4
[junit4:junit4] 2> 1127633 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10049 (-1438320391037648896)]} 0 1
[junit4:junit4] 2> 1127651 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10050 (1438320391053377536)]} 0 5
[junit4:junit4] 2> 1127667 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10050 (1438320391071203328)]} 0 2
[junit4:junit4] 2> 1127714 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10051 (1438320391120486400)]} 0 3
[junit4:junit4] 2> 1127721 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[51 (-1438320391128875008)]} 0 1
[junit4:junit4] 2> 1127727 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[51 (-1438320391136215040)]} 0 1
[junit4:junit4] 2> 1127735 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10051 (1438320391139360768)]} 0 5
[junit4:junit4] 2> 1127742 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1438320391149846528)]} 0 3
[junit4:junit4] 2> 1127754 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1438320391161380864)]} 0 4
[junit4:junit4] 2> 1127829 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438320391241072640)]} 0 2
[junit4:junit4] 2> 1127836 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10051 (-1438320391250509824)]} 0 1
[junit4:junit4] 2> 1127838 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1438320391250509824)]} 0 2
[junit4:junit4] 2> 1127850 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10051 (-1438320391264141312)]} 0 2
[junit4:junit4] 2> 1127861 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10052 (1438320391275675648)]} 0 1
[junit4:junit4] 2> 1127871 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10052 (1438320391286161408)]} 0 1
[junit4:junit4] 2> 1127920 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438320391336493056)]} 0 3
[junit4:junit4] 2> 1127964 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1438320391382630400)]} 0 2
[junit4:junit4] 2> 1127974 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10053 (1438320391394164736)]} 0 1
[junit4:junit4] 2> 1128034 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[54 (-1438320391458127872)]} 0 1
[junit4:junit4] 2> 1128049 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10053 (1438320391437156352)]} 0 36
[junit4:junit4] 2> 1128056 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[54 (-1438320391481196544)]} 0 0
[junit4:junit4] 2> 1128069 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438320391493779456)]} 0 2
[junit4:junit4] 2> 1128078 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1438320391502168064)]} 0 2
[junit4:junit4] 2> 1128115 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1438320391536771072)]} 0 7
[junit4:junit4] 2> 1128119 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10054 (1438320391536771073)]} 0 10
[junit4:junit4] 2> 1128133 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1438320391560888320)]} 0 2
[junit4:junit4] 2> 1128145 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10054 (1438320391572422656)]} 0 2
[junit4:junit4] 2> 1128174 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1438320391602831360)]} 0 2
[junit4:junit4] 2> 1128207 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1438320391638482944)]} 0 2
[junit4:junit4] 2> 1128222 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10054 (-1438320391655260160)]} 0 1
[junit4:junit4] 2> 1128243 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10054 (-1438320391676231680)]} 0 1
[junit4:junit4] 2> 1128269 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10055 (1438320391685668864)]} 0 19
[junit4:junit4] 2> 1128278 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10055 (1438320391711883264)]} 0 3
[junit4:junit4] 2> 1128288 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10056 (1438320391722369024)]} 0 2
[junit4:junit4] 2> 1128298 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1128315 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49361__collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1128316 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10056 (1438320391752777728)]} 0 1
[junit4:junit4] 2> 1128320 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438320391750680576)]} 0 7
[junit4:junit4] 2> 1128349 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1438320391778992128)]} 0 10
[junit4:junit4] 2> 1128357 T1301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1128358 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1128358 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1128371 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1128394 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10057 (1438320391829323776)]} 0 6
[junit4:junit4] 2> 1128405 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10057 (1438320391844003840)]} 0 4
[junit4:junit4] 2> 1128507 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[57 (-1438320391953055744)]} 0 2
[junit4:junit4] 2> 1128515 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10058 (1438320391960395776)]} 0 2
[junit4:junit4] 2> 1128519 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[57 (-1438320391966687232)]} 0 1
[junit4:junit4] 2> 1128529 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438320391975075840)]} 0 2
[junit4:junit4] 2> 1128535 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10058 (1438320391982415872)]} 0 2
[junit4:junit4] 2> 1128542 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1438320391988707328)]} 0 2
[junit4:junit4] 2> 1128575 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438320392024358912)]} 0 2
[junit4:junit4] 2> 1128592 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1438320392041136128)]} 0 2
[junit4:junit4] 2> 1128633 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10057 (-1438320392086224896)]} 0 1
[junit4:junit4] 2> 1128640 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10057 (-1438320392093564928)]} 0 1
[junit4:junit4] 2> 1128689 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1438320392112439296)]} 0 33
[junit4:junit4] 2> 1128690 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10059 (1438320392120827904)]} 0 25
[junit4:junit4] 2> 1128714 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1438320392169062400)]} 0 3
[junit4:junit4] 2> 1128735 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10059 (1438320392186888192)]} 0 7
[junit4:junit4] 2> 1128770 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10060 (1438320392227782656)]} 0 2
[junit4:junit4] 2> 1128773 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1438320392228831232)]} 0 5
[junit4:junit4] 2> 1128782 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10060 (1438320392241414144)]} 0 5
[junit4:junit4] 2> 1128792 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1438320392248754176)]} 0 4
[junit4:junit4] 2> 1128847 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[59 (-1438320392310620160)]} 0 1
[junit4:junit4] 2> 1128852 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10061 (1438320392313765888)]} 0 3
[junit4:junit4] 2> 1128867 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[59 (-1438320392330543104)]} 0 3
[junit4:junit4] 2> 1128875 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10061 (1438320392337883136)]} 0 4
[junit4:junit4] 2> 1128883 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1438320392346271744)]} 0 2
[junit4:junit4] 2> 1128901 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[63 (1438320392366194688)]} 0 2
[junit4:junit4] 2> 1128936 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10062 (1438320392401846272)]} 0 2
[junit4:junit4] 2> 1128954 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10062 (1438320392420720640)]} 0 4
[junit4:junit4] 2> 1128993 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1438320392461615104)]} 0 2
[junit4:junit4] 2> 1129008 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[64 (1438320392476295168)]} 0 3
[junit4:junit4] 2> 1129042 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10059 (-1438320392515092480)]} 0 0
[junit4:junit4] 2> 1129065 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10059 (-1438320392539209728)]} 0 1
[junit4:junit4] 2> 1129077 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10063 (1438320392549695488)]} 0 2
[junit4:junit4] 2> 1129095 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10063 (1438320392567521280)]} 0 4
[junit4:junit4] 2> 1129096 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[60 (-1438320392571715584)]} 0 1
[junit4:junit4] 2> 1129106 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[60 (-1438320392581152768)]} 0 1
[junit4:junit4] 2> 1129145 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1438320392620998656)]} 0 2
[junit4:junit4] 2> 1129167 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[65 (1438320392638824448)]} 0 7
[junit4:junit4] 2> 1129196 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10064 (1438320392674476032)]} 0 2
[junit4:junit4] 2> 1129204 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10064 (1438320392682864640)]} 0 2
[junit4:junit4] 2> 1129253 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[61 (-1438320392736342016)]} 0 0
[junit4:junit4] 2> 1129261 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[61 (-1438320392744730624)]} 0 0
[junit4:junit4] 2> 1129297 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1438320392780382208)]} 0 15
[junit4:junit4] 2> 1129300 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10060 (-1438320392785625088)]} 0 1
[junit4:junit4] 2> 1129308 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[66 (1438320392791916544)]} 0 3
[junit4:junit4] 2> 1129312 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10060 (-1438320392797159424)]} 0 1
[junit4:junit4] 2> 1129339 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10065 (1438320392814985216)]} 0 11
[junit4:junit4] 2> 1129343 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1438320392829665280)]} 0 2
[junit4:junit4] 2> 1129382 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10065 (1438320392856928256)]} 0 15
[junit4:junit4] 2> 1129406 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[67 (1438320392859025408)]} 0 38
[junit4:junit4] 2> 1129454 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10061 (-1438320392946057216)]} 0 1
[junit4:junit4] 2> 1129467 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10061 (-1438320392959688704)]} 0 4
[junit4:junit4] 2> 1129481 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10066 (1438320392970174464)]} 0 6
[junit4:junit4] 2> 1129497 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10066 (1438320392990097408)]} 0 2
[junit4:junit4] 2> 1129507 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1438320393001631744)]} 0 2
[junit4:junit4] 2> 1129514 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10067 (1438320393010020352)]} 0 1
[junit4:junit4] 2> 1129515 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[68 (1438320393008971776)]} 0 2
[junit4:junit4] 2> 1129526 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10067 (1438320393020506112)]} 0 3
[junit4:junit4] 2> 1129597 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[63 (-1438320393096003584)]} 0 1
[junit4:junit4] 2> 1129602 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[63 (-1438320393101246464)]} 0 1
[junit4:junit4] 2> 1129610 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69 (1438320393107537920)]} 0 3
[junit4:junit4] 2> 1129618 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[69 (1438320393118023680)]} 0 1
[junit4:junit4] 2> 1129622 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10068 (1438320393122217984)]} 0 1
[junit4:junit4] 2> 1129629 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10068 (1438320393129558016)]} 0 2
[junit4:junit4] 2> 1129649 T1219 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49361_
[junit4:junit4] 2> 1129661 T1219 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49361_
[junit4:junit4] 2> 1129674 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[64 (-1438320393177792512)]} 0 1
[junit4:junit4] 2> 1129692 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[64 (-1438320393196666880)]} 0 1
[junit4:junit4] 2> 1129702 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1438320393205055488)]} 0 2
[junit4:junit4] 2> 1129706 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1129715 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[70 (1438320393219735552)]} 0 1
[junit4:junit4] 2> 1129725 T1301 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1129725 T1301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129727 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10063 (-1438320393233367040)]} 0 0
[junit4:junit4] 2> 1129730 T1261 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1129731 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129735 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10063 (-1438320393241755648)]} 0 1
[junit4:junit4] 2> 1129767 T1241 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1129770 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1438320393277407232)]} 0 2
[junit4:junit4] 2> 1129777 T1248 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1129778 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129782 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10069 (1438320393288941568)]} 0 5
[junit4:junit4] 2> 1129839 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10069 (1438320393337176064)]} 0 14
[junit4:junit4] 2> 1129841 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[71 (1438320393340321792)]} 0 32
[junit4:junit4] 2> 1129845 T1302 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1129847 T1302 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1129856 T1302 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1
[junit4:junit4] 2> 1129857 T1302 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1129861 T1302 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1129861 T1302 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1129869 T1302 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/'
[junit4:junit4] 2> 1129871 T1302 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1129872 T1302 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371689203600/collection1/lib/README' to classloader
[junit4:junit4] 2> 1129892 T1233 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10064 (-1438320393406382080)]} 0 1
[junit4:junit4] 2> 1129904 T1256 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[10064 (-1438320393417916416)]} 0 2
[junit4:junit4] 2> 1129921 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1438320393434693632)]} 0 2
[junit4:junit4] 2> 1129924 T1242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1129925 T1235 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10070 (1438320393439936512)]} 0 2
[junit4:junit4] 2> 1129932 T1242 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49361_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49361"}
[junit4:junit4] 2> 1129932 T1242 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 1129936 T1254 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[72 (1438320393445179392)]} 0 8
[junit4:junit4] 2> 1129937 T1252 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10070 (1438320393452519424)]} 0 2
[junit4:junit4] 2> 1129983 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129984 T1241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129988 T1237 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10071 (1438320393505996800)]} 0 2
[junit4:junit4] 2> 1129991 T1261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1129996 T1301 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1130004 T1236 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[73 (1438320393521725440)]} 0 3
[junit4:junit4] 2> 1130009 T1257 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10071 (1438320393526968320)]} 0 2
[junit4:junit4] 2> 1130036 T1255 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[73 (1438320393544794112)]} 0 13
[junit4:junit4] 2> 1130115 T1234 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10072 (1438320393638117376)]} 0 2
[junit4:junit4] 2> 1130131 T1253 C628 P49358 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10072 (1438320393655943168)]} 0 2
[junit4:junit4] 2> 1130133 T1232 C627 P49354 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[74 (1438320393644408832)
[...truncated too long message...]
olr cannot talk to ZK
[junit4:junit4] 2> 3935802 T7170 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 3935805 T7095 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51229
[junit4:junit4] 2> !!!! WARNING: best effort to remove /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371691944572 FAILED !!!!!
[junit4:junit4] 2> 3935806 T7095 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 3935807 T7095 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51226 51226
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=24ACDDE86AA4AFF2 -Dtests.slow=true -Dtests.locale=ar_BH -Dtests.timezone=Pacific/Enderbury -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 79.3s | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51233 returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([24ACDDE86AA4AFF2:A54A53F01DFBCFCE]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 3935817 T7095 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 79325 T7094 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ar_BH, timezone=Pacific/Enderbury
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=368092136,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestStressVersions, SoftAutoCommitTest, TestQueryUtils, TestCloudManagedSchemaAddField, SolrCoreCheckLockOnStartupTest, PolyFieldTest, DirectUpdateHandlerTest, BasicFunctionalityTest, TestPHPSerializedResponseWriter, SampleTest, TestCoreContainer, TestCopyFieldCollectionResource, SolrCoreTest, TestStressRecovery, TestConfig, CSVRequestHandlerTest, UpdateRequestProcessorFactoryTest, AutoCommitTest, LukeRequestHandlerTest, TestRecovery, TestManagedSchema, TestPostingsSolrHighlighter, TestUpdate, BadIndexSchemaTest, TestWriterPerf, TestShardHandlerFactory, TestRandomFaceting, TestSolrDeletionPolicy2, TestFieldCollectionResource, QueryResultKeyTest, PrimUtilsTest, PluginInfoTest, URLClassifyProcessorTest, TestSchemaResource, DateFieldTest, MultiTermTest, TestCollationField, TestIndexingPerformance, TestSolr4Spatial, ResourceLoaderTest, LegacyHTMLStripCharFilterTest, ZkControllerTest, SpellCheckCollatorTest, JsonLoaderTest, DistributedTermsComponentTest, SortByFunctionTest, SuggesterTSTTest, TestSystemIdResolver, ShardSplitTest, TestArbitraryIndexDir, TestSerializedLuceneMatchVersion, TestPluginEnable, TestElisionMultitermQuery, HighlighterTest, TestRealTimeGet, PeerSyncTest, RecoveryZkTest, ZkSolrClientTest, ShardRoutingCustomTest, DocumentAnalysisRequestHandlerTest, TestCodecSupport, TestDocumentBuilder, TestReversedWildcardFilterFactory, SpellCheckComponentTest, RequiredFieldsTest, TestCSVResponseWriter, AliasIntegrationTest, TestReplicationHandler, TestPropInjectDefaults, SliceStateTest, TestManagedSchemaFieldResource, SimplePostToolTest, TestSchemaVersionResource, SearchHandlerTest, TestSolrQueryParser, CachingDirectoryFactoryTest, TestCSVLoader, TestFoldingMultitermQuery, TestJmxMonitoredMap, AlternateDirectoryTest, TestRangeQuery, TestPseudoReturnFields, TestLFUCache, TimeZoneUtilsTest, QueryParsingTest, BasicZkTest, TestPartialUpdateDeduplication, SolrInfoMBeanTest, AnalysisAfterCoreReloadTest, TestPhraseSuggestions, TestPropInject, TestReloadAndDeleteDocs, TestFieldTypeResource, TestRTGBase, TestUtils, IndexBasedSpellCheckerTest, TestFiltering, ShardRoutingTest, TestClassNameShortening, DirectSolrSpellCheckerTest, RAMDirectoryFactoryTest, TestSolrXml, StatsComponentTest, TestValueSourceCache, SuggesterTest, SuggesterFSTTest, TestSweetSpotSimilarityFactory, JSONWriterTest, OverseerCollectionProcessorTest, ReturnFieldsTest, XsltUpdateRequestHandlerTest, QueryElevationComponentTest, TestLuceneMatchVersion, LeaderElectionTest, TestStressLucene, TestDefaultSimilarityFactory, TestTrie, TestWordDelimiterFilterFactory, ClusterStateTest, DocumentBuilderTest, TestRemoteStreaming, UpdateParamsTest, SolrIndexConfigTest, SOLR749Test, ConvertedLegacyTest, EchoParamsTest, DocValuesTest, SystemInfoHandlerTest, SolrIndexSplitterTest, CircularListTest, TestMergePolicyConfig, MBeansHandlerTest, SolrRequestParserTest, SliceStateUpdateTest, TestSolrDeletionPolicy1, TestSolrQueryParserResource, IndexSchemaTest, DistanceFunctionTest, IndexReaderFactoryTest, TestSolrCoreProperties, SimpleFacetsTest, TestSchemaNameResource, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestZkChroot, TestRandomDVFaceting, ZkCLITest, TestDistributedSearch, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestMultiCoreConfBootstrap, TestReload, HardAutoCommitTest, TestGroupingSearch, SolrCmdDistributorTest, TestSort, TestFunctionQuery, TestLazyCores, TestBadConfig, TestIndexSearcher, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, CoreAdminHandlerTest, SpatialFilterTest, SuggesterWFSTTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, DocValuesMultiTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, TestComponentsName, CoreContainerCoreInitFailuresTest, TestBinaryResponseWriter, HighlighterConfigTest, TestQuerySenderListener, TestSolrIndexConfig, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, BadComponentTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestNumberUtils, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, TestSolrXMLSerializer, PreAnalyzedFieldTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, UUIDFieldTest, FileUtilsTest, SolrTestCaseJ4Test, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed in 79.63s, 1 test, 1 error <<< FAILURES!
[...truncated 55 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 1 failure, 18 ignored (6 assumptions)
Total time: 117 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 577 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/577/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:49422/collection1lastClient and got 261 from http://127.0.0.1:49429/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49422/collection1lastClient and got 261 from http://127.0.0.1:49429/collection1
at __randomizedtesting.SeedInfo.seed([3953250FFE0FF89E:B8B5AB17895098A2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9209 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 565641 T1030 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 565649 T1030 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371662335566
[junit4:junit4] 2> 565651 T1030 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 565653 T1031 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 565754 T1030 oasc.ZkTestServer.run start zk server on port:49415
[junit4:junit4] 2> 565757 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 565763 T1037 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c27b65 name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 565763 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 565763 T1030 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 565777 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 565786 T1039 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4fd999dd name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 565787 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 565787 T1030 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 565795 T1030 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 565802 T1030 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 565808 T1030 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 565819 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 565825 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 565838 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 565839 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 565849 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 565851 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 565858 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 565860 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 565867 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 565868 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 565877 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 565878 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 565888 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 565894 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 565903 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 565905 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 565916 T1030 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 565917 T1030 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 566455 T1030 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 566468 T1030 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49418
[junit4:junit4] 2> 566469 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 566470 T1030 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 566471 T1030 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842
[junit4:junit4] 2> 566494 T1030 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/solr.xml
[junit4:junit4] 2> 566495 T1030 oasc.CoreContainer.<init> New CoreContainer 1475052298
[junit4:junit4] 2> 566496 T1030 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/'
[junit4:junit4] 2> 566496 T1030 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/'
[junit4:junit4] 2> 566698 T1030 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 566700 T1030 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 566700 T1030 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 566701 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 566702 T1030 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 566702 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 566703 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 566703 T1030 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 566704 T1030 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 566705 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 566734 T1030 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 566736 T1030 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49415/solr
[junit4:junit4] 2> 566737 T1030 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 566740 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 566769 T1050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@587dccab name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 566770 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 566778 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 566823 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 566824 T1052 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ed7793b name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 566825 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 566839 T1030 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 566849 T1030 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 566859 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 566865 T1030 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49418_
[junit4:junit4] 2> 566868 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49418_
[junit4:junit4] 2> 566878 T1030 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 566901 T1030 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 566909 T1030 oasc.Overseer.start Overseer (id=89893262824439811-127.0.0.1:49418_-n_0000000000) starting
[junit4:junit4] 2> 566918 T1030 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 566931 T1054 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 566934 T1030 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 566943 T1030 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 566948 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 566970 T1053 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 566972 T1055 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 566972 T1055 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 568484 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 568487 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49418_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49418"}
[junit4:junit4] 2> 568487 T1053 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 568488 T1053 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 568509 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 568978 T1055 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/collection1
[junit4:junit4] 2> 568979 T1055 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 568981 T1055 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 568982 T1055 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 568986 T1055 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/collection1/'
[junit4:junit4] 2> 568989 T1055 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 568991 T1055 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/collection1/lib/README' to classloader
[junit4:junit4] 2> 569085 T1055 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 569205 T1055 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 569213 T1055 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 569221 T1055 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 570575 T1055 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 570601 T1055 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 570609 T1055 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 570659 T1055 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 570675 T1055 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 570685 T1055 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 570689 T1055 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 570690 T1055 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 570690 T1055 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 570694 T1055 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 570695 T1055 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 570695 T1055 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 570767 T1055 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371662335842/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/
[junit4:junit4] 2> 570768 T1055 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c3a0d86
[junit4:junit4] 2> 570769 T1055 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 570771 T1055 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data
[junit4:junit4] 2> 570772 T1055 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index/
[junit4:junit4] 2> 570774 T1055 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 570777 T1055 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index
[junit4:junit4] 2> 570795 T1055 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 570796 T1055 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 570802 T1055 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 570803 T1055 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 570804 T1055 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 570805 T1055 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 570806 T1055 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 570807 T1055 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 570807 T1055 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 570808 T1055 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 570809 T1055 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 570830 T1055 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 570844 T1055 oass.SolrIndexSearcher.<init> Opening Searcher@d107335 main
[junit4:junit4] 2> 570846 T1055 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 570846 T1055 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 570855 T1055 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 570855 T1055 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49418 collection:control_collection shard:shard1
[junit4:junit4] 2> 570858 T1056 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d107335 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 570865 T1055 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 570887 T1055 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 570895 T1055 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 570896 T1055 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 570896 T1055 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49418/collection1/
[junit4:junit4] 2> 570897 T1055 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 570897 T1055 oasc.SyncStrategy.syncToMe http://127.0.0.1:49418/collection1/ has no replicas
[junit4:junit4] 2> 570897 T1055 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49418/collection1/
[junit4:junit4] 2> 570898 T1055 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 571551 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 571578 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 571605 T1055 oasc.ZkController.register We are http://127.0.0.1:49418/collection1/ and leader is http://127.0.0.1:49418/collection1/
[junit4:junit4] 2> 571605 T1055 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49418
[junit4:junit4] 2> 571606 T1055 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 571606 T1055 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 571607 T1055 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 571610 T1055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 571614 T1030 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 571614 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 571615 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 571651 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 571680 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 571688 T1059 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9e3abb6 name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 571690 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 571698 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 571706 T1030 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 572275 T1030 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 572288 T1030 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49422
[junit4:junit4] 2> 572291 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 572294 T1030 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 572295 T1030 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622
[junit4:junit4] 2> 572296 T1030 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/solr.xml
[junit4:junit4] 2> 572296 T1030 oasc.CoreContainer.<init> New CoreContainer 2103028239
[junit4:junit4] 2> 572297 T1030 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/'
[junit4:junit4] 2> 572298 T1030 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/'
[junit4:junit4] 2> 572506 T1030 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 572507 T1030 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 572508 T1030 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 572508 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 572509 T1030 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 572510 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 572510 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 572511 T1030 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 572512 T1030 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 572512 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 572545 T1030 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 572547 T1030 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49415/solr
[junit4:junit4] 2> 572547 T1030 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 572556 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 572561 T1070 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11fdc5a name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 572562 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 572571 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 572597 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 572604 T1072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1da37927 name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 572605 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 572625 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 573100 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 573102 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49418__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49418_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49418"}
[junit4:junit4] 2> 573112 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 573113 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 573113 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 573643 T1030 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49422_
[junit4:junit4] 2> 573646 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49422_
[junit4:junit4] 2> 573652 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 573653 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 573662 T1059 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 573663 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 573664 T1072 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 573666 T1052 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 573682 T1073 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 573682 T1073 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 574630 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 574635 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49422_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49422"}
[junit4:junit4] 2> 574638 T1053 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 574639 T1053 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 574649 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 574650 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 574649 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 574693 T1073 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/collection1
[junit4:junit4] 2> 574694 T1073 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 574695 T1073 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 574696 T1073 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 574700 T1073 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/collection1/'
[junit4:junit4] 2> 574704 T1073 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 574705 T1073 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/collection1/lib/README' to classloader
[junit4:junit4] 2> 574793 T1073 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 574923 T1073 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 574928 T1073 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 574939 T1073 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 576047 T1073 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 576065 T1073 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 576071 T1073 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 576117 T1073 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 576126 T1073 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 576134 T1073 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 576136 T1073 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 576138 T1073 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 576139 T1073 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 576141 T1073 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 576142 T1073 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 576142 T1073 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 576143 T1073 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371662341622/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/
[junit4:junit4] 2> 576143 T1073 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c3a0d86
[junit4:junit4] 2> 576144 T1073 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 576146 T1073 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1
[junit4:junit4] 2> 576147 T1073 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index/
[junit4:junit4] 2> 576148 T1073 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 576149 T1073 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index
[junit4:junit4] 2> 576162 T1073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 576163 T1073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 576170 T1073 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 576171 T1073 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 576172 T1073 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 576173 T1073 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 576175 T1073 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 576175 T1073 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 576176 T1073 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 576177 T1073 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 576178 T1073 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 576198 T1073 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 576210 T1073 oass.SolrIndexSearcher.<init> Opening Searcher@17e9846d main
[junit4:junit4] 2> 576212 T1073 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 576213 T1073 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 576233 T1073 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 576234 T1073 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49422 collection:collection1 shard:shard1
[junit4:junit4] 2> 576234 T1074 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17e9846d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 576235 T1073 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 576253 T1073 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 576260 T1073 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 576260 T1073 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 576261 T1073 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 576261 T1073 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 576262 T1073 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/ has no replicas
[junit4:junit4] 2> 576262 T1073 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 576262 T1073 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 577669 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 577687 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 577690 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 577694 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 577714 T1073 oasc.ZkController.register We are http://127.0.0.1:49422/collection1/ and leader is http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 577715 T1073 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49422
[junit4:junit4] 2> 577715 T1073 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 577716 T1073 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 577716 T1073 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 577720 T1073 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 577724 T1030 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 577726 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 577727 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 579072 T1030 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 579090 T1030 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49425
[junit4:junit4] 2> 579091 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 579094 T1030 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 579095 T1030 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696
[junit4:junit4] 2> 579096 T1030 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/solr.xml
[junit4:junit4] 2> 579097 T1030 oasc.CoreContainer.<init> New CoreContainer 1107403451
[junit4:junit4] 2> 579103 T1030 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/'
[junit4:junit4] 2> 579103 T1030 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/'
[junit4:junit4] 2> 579225 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 579227 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49422__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49422_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49422"}
[junit4:junit4] 2> 579238 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 579239 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 579239 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 579423 T1030 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 579424 T1030 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 579425 T1030 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 579425 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 579426 T1030 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 579427 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 579428 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 579430 T1030 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 579431 T1030 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 579432 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 579503 T1030 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 579504 T1030 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49415/solr
[junit4:junit4] 2> 579505 T1030 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 579509 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 579512 T1086 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1955e3f0 name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 579513 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 579530 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 579606 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 579613 T1088 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@283d1416 name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 579613 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 579644 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 580672 T1030 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49425_
[junit4:junit4] 2> 580678 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49425_
[junit4:junit4] 2> 580690 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 580691 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 580694 T1059 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 580694 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 580698 T1088 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 580713 T1089 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 580714 T1089 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 580723 T1072 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 580731 T1052 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 580758 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 580762 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49425"}
[junit4:junit4] 2> 580763 T1053 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 580764 T1053 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 580780 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 580781 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 580785 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 580790 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 581719 T1089 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/collection1
[junit4:junit4] 2> 581720 T1089 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 581722 T1089 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 581722 T1089 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 581732 T1089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/collection1/'
[junit4:junit4] 2> 581734 T1089 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 581735 T1089 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/collection1/lib/README' to classloader
[junit4:junit4] 2> 581833 T1089 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 581941 T1089 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 581945 T1089 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 581953 T1089 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 583157 T1089 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 583178 T1089 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 583186 T1089 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 583262 T1089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 583278 T1089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 583289 T1089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 583314 T1089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 583315 T1089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 583316 T1089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 583320 T1089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 583321 T1089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 583321 T1089 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 583322 T1089 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371662347696/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/
[junit4:junit4] 2> 583322 T1089 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c3a0d86
[junit4:junit4] 2> 583323 T1089 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 583325 T1089 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2
[junit4:junit4] 2> 583326 T1089 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index/
[junit4:junit4] 2> 583327 T1089 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 583328 T1089 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index
[junit4:junit4] 2> 583342 T1089 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 583343 T1089 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 583348 T1089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 583348 T1089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 583349 T1089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 583353 T1089 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 583354 T1089 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 583354 T1089 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 583355 T1089 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 583356 T1089 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 583357 T1089 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 583377 T1089 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 583492 T1089 oass.SolrIndexSearcher.<init> Opening Searcher@594b729e main
[junit4:junit4] 2> 583494 T1089 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 583494 T1089 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 583503 T1090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@594b729e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 583506 T1089 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 583506 T1089 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49425 collection:collection1 shard:shard1
[junit4:junit4] 2> 583514 T1089 oasc.ZkController.register We are http://127.0.0.1:49425/collection1/ and leader is http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 583515 T1089 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49425
[junit4:junit4] 2> 583515 T1089 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 583515 T1089 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C263 name=collection1 org.apache.solr.core.SolrCore@69c806c url=http://127.0.0.1:49425/collection1 node=127.0.0.1:49425_ C263_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49425_, base_url=http://127.0.0.1:49425}
[junit4:junit4] 2> 583517 T1091 C263 P49425 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 583518 T1089 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 583523 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 583524 T1091 C263 P49425 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 583525 T1091 C263 P49425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 583528 T1091 C263 P49425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 583530 T1030 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 583538 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 583539 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 583583 T1067 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49425__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 583857 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 583860 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49425__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49425"}
[junit4:junit4] 2> 583887 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 583887 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 583889 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 583889 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 584184 T1030 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 584198 T1030 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49429
[junit4:junit4] 2> 584201 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 584202 T1030 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 584202 T1030 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482
[junit4:junit4] 2> 584203 T1030 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/solr.xml
[junit4:junit4] 2> 584204 T1030 oasc.CoreContainer.<init> New CoreContainer 798737610
[junit4:junit4] 2> 584204 T1030 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/'
[junit4:junit4] 2> 584205 T1030 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/'
[junit4:junit4] 2> 584440 T1030 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 584441 T1030 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 584441 T1030 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 584442 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 584443 T1030 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 584444 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 584444 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 584445 T1030 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 584446 T1030 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 584461 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 584494 T1030 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 584495 T1030 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49415/solr
[junit4:junit4] 2> 584496 T1030 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 584502 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 584585 T1067 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49425__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 584586 T1067 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49425__collection1&state=recovering&nodeName=127.0.0.1:49425_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 584640 T1103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72077d0 name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 584641 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 584710 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 584718 T1032 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f5d723e5a0009, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 584739 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 584745 T1105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d27c901 name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 584745 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 584757 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 585772 T1030 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49429_
[junit4:junit4] 2> 585778 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49429_
[junit4:junit4] 2> 585784 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 585785 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 585789 T1059 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 585789 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 585795 T1072 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 585800 T1088 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 585800 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 585801 T1105 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 585818 T1106 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 585819 T1106 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 585821 T1052 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> C263_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49425_, base_url=http://127.0.0.1:49425}
[junit4:junit4] 2> 586590 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49422/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 586591 T1091 C263 P49425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49425 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 586593 T1091 C263 P49425 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 586602 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 586603 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 586603 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 586604 T1091 C263 P49425 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 586604 T1091 C263 P49425 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49422/collection1/. core=collection1
[junit4:junit4] 2> 586605 T1091 C263 P49425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C264 name=collection1 org.apache.solr.core.SolrCore@15784174 url=http://127.0.0.1:49422/collection1 node=127.0.0.1:49422_ C264_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49422_, base_url=http://127.0.0.1:49422, leader=true}
[junit4:junit4] 2> 586644 T1064 C264 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 586651 T1065 C264 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 586665 T1065 C264 P49422 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 586667 T1065 C264 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 586672 T1065 C264 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 586673 T1065 C264 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 586675 T1065 C264 P49422 oass.SolrIndexSearcher.<init> Opening Searcher@306cd715 realtime
[junit4:junit4] 2> 586676 T1065 C264 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 586677 T1065 C264 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 586682 T1091 C263 P49425 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 586683 T1091 C263 P49425 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 586692 T1068 C264 P49422 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 586693 T1068 C264 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=4
[junit4:junit4] 2> 586695 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 586695 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 586695 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 586700 T1063 C264 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 586708 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 586711 T1091 C263 P49425 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624
[junit4:junit4] 2> 586713 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2171890d fullCopy=false
[junit4:junit4] 2> 586719 T1067 C264 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 586728 T1091 C263 P49425 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 586735 T1091 C263 P49425 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 586736 T1091 C263 P49425 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 586742 T1091 C263 P49425 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 586745 T1091 C263 P49425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 586746 T1091 C263 P49425 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 586747 T1091 C263 P49425 oass.SolrIndexSearcher.<init> Opening Searcher@65ee31d1 main
[junit4:junit4] 2> 586749 T1090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65ee31d1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 586751 T1091 C263 P49425 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624;done=true>>]
[junit4:junit4] 2> 586752 T1091 C263 P49425 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624
[junit4:junit4] 2> 586752 T1091 C263 P49425 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index.20130620051916624
[junit4:junit4] 2> 586754 T1091 C263 P49425 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 586754 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 586755 T1091 C263 P49425 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 586755 T1091 C263 P49425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 586778 T1091 C263 P49425 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 586924 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 586926 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49429"}
[junit4:junit4] 2> 586926 T1053 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 586926 T1053 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 586939 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49425__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49425"}
[junit4:junit4] 2> 586950 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 586951 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 586950 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 586952 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 586954 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 587825 T1106 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/collection1
[junit4:junit4] 2> 587827 T1106 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 587830 T1106 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 587831 T1106 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 587837 T1106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/collection1/'
[junit4:junit4] 2> 587839 T1106 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 587840 T1106 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/collection1/lib/README' to classloader
[junit4:junit4] 2> 587979 T1106 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 588082 T1106 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 588088 T1106 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 588097 T1106 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 589266 T1106 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 589287 T1106 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 589293 T1106 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 589352 T1106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 589363 T1106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 589373 T1106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 589378 T1106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 589379 T1106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 589379 T1106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 589383 T1106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 589385 T1106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 589385 T1106 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 589386 T1106 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371662353482/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/
[junit4:junit4] 2> 589386 T1106 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c3a0d86
[junit4:junit4] 2> 589387 T1106 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 589389 T1106 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3
[junit4:junit4] 2> 589390 T1106 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index/
[junit4:junit4] 2> 589391 T1106 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 589392 T1106 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index
[junit4:junit4] 2> 589401 T1106 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 589401 T1106 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 589407 T1106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 589408 T1106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 589409 T1106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 589411 T1106 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 589412 T1106 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 589412 T1106 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 589418 T1106 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 589419 T1106 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 589420 T1106 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 589445 T1106 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 589459 T1106 oass.SolrIndexSearcher.<init> Opening Searcher@47dc6329 main
[junit4:junit4] 2> 589462 T1106 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 589462 T1106 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 589474 T1109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47dc6329 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 589478 T1106 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 589478 T1106 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49429 collection:collection1 shard:shard1
[junit4:junit4] 2> 589488 T1106 oasc.ZkController.register We are http://127.0.0.1:49429/collection1/ and leader is http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 589489 T1106 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49429
[junit4:junit4] 2> 589489 T1106 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 589489 T1106 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C265 name=collection1 org.apache.solr.core.SolrCore@1188c915 url=http://127.0.0.1:49429/collection1 node=127.0.0.1:49429_ C265_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49429_, base_url=http://127.0.0.1:49429}
[junit4:junit4] 2> 589491 T1110 C265 P49429 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 589493 T1106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 589499 T1030 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 589500 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 589501 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 589530 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 589530 T1110 C265 P49429 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 589532 T1110 C265 P49429 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 589548 T1110 C265 P49429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 589592 T1065 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49429__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 590068 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 590074 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49429__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49429"}
[junit4:junit4] 2> 590109 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 590109 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 590110 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 590110 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 590111 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 590276 T1030 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 590287 T1030 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49436
[junit4:junit4] 2> 590288 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 590288 T1030 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 590289 T1030 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468
[junit4:junit4] 2> 590290 T1030 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/solr.xml
[junit4:junit4] 2> 590291 T1030 oasc.CoreContainer.<init> New CoreContainer 2000835339
[junit4:junit4] 2> 590293 T1030 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/'
[junit4:junit4] 2> 590294 T1030 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/'
[junit4:junit4] 2> 590572 T1030 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 590573 T1030 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 590574 T1030 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 590575 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 590575 T1030 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 590576 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 590578 T1030 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 590578 T1030 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 590579 T1030 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 590580 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 590594 T1065 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49429__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 590595 T1065 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49429__collection1&state=recovering&nodeName=127.0.0.1:49429_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> 590615 T1030 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 590619 T1030 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49415/solr
[junit4:junit4] 2> 590626 T1030 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 590628 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 590633 T1122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c5371b1 name:ZooKeeperConnection Watcher:127.0.0.1:49415 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 590635 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 590641 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 590666 T1030 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 590673 T1124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38ccc094 name:ZooKeeperConnection Watcher:127.0.0.1:49415/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 590674 T1030 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 590692 T1030 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 591704 T1030 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49436_
[junit4:junit4] 2> 591708 T1030 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49436_
[junit4:junit4] 2> 591715 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 591726 T1088 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591727 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 591728 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 591727 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591729 T1124 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591733 T1059 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591734 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 591738 T1052 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591741 T1072 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591744 T1105 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 591756 T1125 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 591757 T1125 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C265_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49429_, base_url=http://127.0.0.1:49429}
[junit4:junit4] 2> 592597 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49422/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 592598 T1110 C265 P49429 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49429 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 592600 T1110 C265 P49429 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 592600 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 592601 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 592601 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 592602 T1110 C265 P49429 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 592602 T1110 C265 P49429 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49422/collection1/. core=collection1
[junit4:junit4] 2> 592603 T1110 C265 P49429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C266 name=collection1 org.apache.solr.core.SolrCore@15784174 url=http://127.0.0.1:49422/collection1 node=127.0.0.1:49422_ C266_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49422_, base_url=http://127.0.0.1:49422, leader=true}
[junit4:junit4] 2> 592641 T1063 C266 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 592649 T1067 C266 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 592658 T1067 C266 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 592659 T1067 C266 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 592662 T1067 C266 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 592663 T1067 C266 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 592666 T1110 C265 P49429 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 592668 T1110 C265 P49429 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 592678 T1066 C266 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 592681 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 592681 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 592682 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 592687 T1068 C266 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=2
[junit4:junit4] 2> 592689 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 592692 T1110 C265 P49429 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605
[junit4:junit4] 2> 592695 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7461d8f5 fullCopy=false
[junit4:junit4] 2> 592701 T1065 C266 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1
[junit4:junit4] 2> 592706 T1110 C265 P49429 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 592721 T1110 C265 P49429 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 592721 T1110 C265 P49429 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 592732 T1110 C265 P49429 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 592734 T1110 C265 P49429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 592735 T1110 C265 P49429 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 592737 T1110 C265 P49429 oass.SolrIndexSearcher.<init> Opening Searcher@64ad7efb main
[junit4:junit4] 2> 592741 T1109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64ad7efb main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 592742 T1110 C265 P49429 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605;done=true>>]
[junit4:junit4] 2> 592743 T1110 C265 P49429 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605
[junit4:junit4] 2> 592743 T1110 C265 P49429 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index.20130620051922605
[junit4:junit4] 2> 592746 T1110 C265 P49429 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 592747 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 592747 T1110 C265 P49429 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 592748 T1110 C265 P49429 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 592752 T1110 C265 P49429 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 593148 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 593151 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49436_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49436"}
[junit4:junit4] 2> 593151 T1053 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 593151 T1053 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 593167 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49429__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49429"}
[junit4:junit4] 2> 593192 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593193 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593197 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593198 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593199 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593199 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 593767 T1125 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/collection1
[junit4:junit4] 2> 593768 T1125 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 593770 T1125 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 593771 T1125 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 593778 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/collection1/'
[junit4:junit4] 2> 593780 T1125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 593781 T1125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/collection1/lib/README' to classloader
[junit4:junit4] 2> 593955 T1125 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 594068 T1125 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 594072 T1125 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 594092 T1125 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 595375 T1125 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 595493 T1125 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 595505 T1125 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 595576 T1125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 595591 T1125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 595603 T1125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 595608 T1125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 595610 T1125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 595610 T1125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 595616 T1125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 595616 T1125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 595617 T1125 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 595618 T1125 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371662359468/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/
[junit4:junit4] 2> 595618 T1125 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c3a0d86
[junit4:junit4] 2> 595619 T1125 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 595621 T1125 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4
[junit4:junit4] 2> 595623 T1125 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index/
[junit4:junit4] 2> 595624 T1125 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 595626 T1125 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index
[junit4:junit4] 2> 595642 T1125 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 595642 T1125 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 595652 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 595653 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 595654 T1125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 595656 T1125 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 595658 T1125 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 595659 T1125 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 595664 T1125 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 595666 T1125 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 595673 T1125 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 595702 T1125 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 595721 T1125 oass.SolrIndexSearcher.<init> Opening Searcher@1f1ac96c main
[junit4:junit4] 2> 595723 T1125 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 595724 T1125 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 595736 T1125 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 595737 T1125 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49436 collection:collection1 shard:shard1
[junit4:junit4] 2> 595740 T1128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f1ac96c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 595751 T1125 oasc.ZkController.register We are http://127.0.0.1:49436/collection1/ and leader is http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 595752 T1125 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49436
[junit4:junit4] 2> 595754 T1125 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 595756 T1125 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C267 name=collection1 org.apache.solr.core.SolrCore@158167d url=http://127.0.0.1:49436/collection1 node=127.0.0.1:49436_ C267_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49436_, base_url=http://127.0.0.1:49436}
[junit4:junit4] 2> 595761 T1129 C267 P49436 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 595763 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 595764 T1129 C267 P49436 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 595765 T1129 C267 P49436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 595764 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 595767 T1129 C267 P49436 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 595769 T1030 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 595770 T1030 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 595771 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 595805 T1067 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49436__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 595806 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 595808 T1030 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 595809 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 595811 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 596244 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 596246 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49436__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49436_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49436"}
[junit4:junit4] 2> 596256 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596256 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596257 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596262 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596264 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596272 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 596807 T1067 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:49436__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 596808 T1067 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:49436__collection1&state=recovering&nodeName=127.0.0.1:49436_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 596818 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 597829 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C267_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49436_, base_url=http://127.0.0.1:49436}
[junit4:junit4] 2> 598813 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49422/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 598814 T1129 C267 P49436 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49436 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 598815 T1129 C267 P49436 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 598815 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 598815 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 598816 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 598816 T1129 C267 P49436 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 598817 T1129 C267 P49436 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49422/collection1/. core=collection1
[junit4:junit4] 2> 598817 T1129 C267 P49436 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C268 name=collection1 org.apache.solr.core.SolrCore@15784174 url=http://127.0.0.1:49422/collection1 node=127.0.0.1:49422_ C268_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49422_, base_url=http://127.0.0.1:49422, leader=true}
[junit4:junit4] 2> 598818 T1067 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 598850 T1065 C268 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 598858 T1065 C268 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 598861 T1065 C268 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 598866 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 598869 T1065 C268 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 598870 T1065 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 598872 T1129 C267 P49436 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 598872 T1129 C267 P49436 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 598876 T1064 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 598880 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 598881 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 598881 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 598886 T1066 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 598888 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 598891 T1129 C267 P49436 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804
[junit4:junit4] 2> 598893 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804 lockFactory=org.apache.lucene.store.NativeFSLockFactory@288f3faa fullCopy=false
[junit4:junit4] 2> 598898 T1067 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 598902 T1129 C267 P49436 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 598907 T1129 C267 P49436 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 598908 T1129 C267 P49436 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 598915 T1129 C267 P49436 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 598918 T1129 C267 P49436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 598919 T1129 C267 P49436 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 598920 T1129 C267 P49436 oass.SolrIndexSearcher.<init> Opening Searcher@6c66b522 main
[junit4:junit4] 2> 598922 T1128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c66b522 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 598924 T1129 C267 P49436 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804;done=true>>]
[junit4:junit4] 2> 598925 T1129 C267 P49436 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804
[junit4:junit4] 2> 598927 T1129 C267 P49436 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index.20130620051928804
[junit4:junit4] 2> 598934 T1129 C267 P49436 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 598935 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 598936 T1129 C267 P49436 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 598936 T1129 C267 P49436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 598940 T1129 C267 P49436 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 599284 T1053 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 599286 T1053 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49436__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49436_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49436"}
[junit4:junit4] 2> 599299 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599300 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599305 T1052 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599309 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599310 T1059 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599311 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 599875 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 599878 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C269 name=collection1 org.apache.solr.core.SolrCore@1f609b02 url=http://127.0.0.1:49418/collection1 node=127.0.0.1:49418_ C269_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49418_, base_url=http://127.0.0.1:49418, leader=true}
[junit4:junit4] 2> 599922 T1045 C269 P49418 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 599938 T1045 C269 P49418 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 599941 T1045 C269 P49418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 599950 T1045 C269 P49418 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 599953 T1045 C269 P49418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 599956 T1045 C269 P49418 oass.SolrIndexSearcher.<init> Opening Searcher@15063069 main
[junit4:junit4] 2> 599957 T1045 C269 P49418 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 599959 T1056 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15063069 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 599960 T1045 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4] 2> ASYNC NEW_CORE C270 name=collection1 org.apache.solr.core.SolrCore@1188c915 url=http://127.0.0.1:49429/collection1 node=127.0.0.1:49429_ C270_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49429_, base_url=http://127.0.0.1:49429}
[junit4:junit4] 2> 600076 T1100 C270 P49429 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49422/collection1/, StdNode: http://127.0.0.1:49425/collection1/, StdNode: http://127.0.0.1:49429/collection1/, StdNode: http://127.0.0.1:49436/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> C267_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49436_, base_url=http://127.0.0.1:49436}
[junit4:junit4] 2> 600138 T1116 C267 P49436 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 600146 T1066 C268 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 600159 T1099 C270 P49429 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 600151 T1116 C267 P49436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 600170 T1116 C267 P49436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 600183 T1066 C268 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 600185 T1066 C268 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 600188 T1066 C268 P49422 oass.SolrIndexSearcher.<init> Opening Searcher@12a0b95d main
[junit4:junit4] 2> 600189 T1066 C268 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 600197 T1074 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12a0b95d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 600198 T1066 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 52
[junit4:junit4] 2> 600219 T1116 C267 P49436 oass.SolrIndexSearcher.<init> Opening Searcher@12d93aef main
[junit4:junit4] 2> 600219 T1116 C267 P49436 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 600224 T1128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12d93aef main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 600229 T1099 C270 P49429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 600229 T1099 C270 P49429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 600252 T1099 C270 P49429 oass.SolrIndexSearcher.<init> Opening Searcher@3cd280d0 main
[junit4:junit4] 2> 600255 T1109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd280d0 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 600255 T1099 C270 P49429 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 600256 T1099 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 97
[junit4:junit4] 2> 600259 T1116 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 121
[junit4:junit4] 2> ASYNC NEW_CORE C271 name=collection1 org.apache.solr.core.SolrCore@69c806c url=http://127.0.0.1:49425/collection1 node=127.0.0.1:49425_ C271_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49425_, base_url=http://127.0.0.1:49425}
[junit4:junit4] 2> 600261 T1080 C271 P49425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 600269 T1080 C271 P49425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 600274 T1080 C271 P49425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 600278 T1080 C271 P49425 oass.SolrIndexSearcher.<init> Opening Searcher@65318dd6 main
[junit4:junit4] 2> 600279 T1080 C271 P49425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 600281 T1090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65318dd6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 600282 T1080 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 34
[junit4:junit4] 2> 600286 T1100 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 211
[junit4:junit4] 2> 600294 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 600302 T1067 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 600309 T1082 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 600316 T1101 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 600323 T1118 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 602344 T1045 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438292243651756032)} 0 8
[junit4:junit4] 2> 602397 T1097 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438292243669581824&update.from=http://127.0.0.1:49422/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438292243669581824)} 0 4
[junit4:junit4] 2> 602406 T1118 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438292243669581824&update.from=http://127.0.0.1:49422/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438292243669581824)} 0 3
[junit4:junit4] 2> 602417 T1082 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438292243669581824&update.from=http://127.0.0.1:49422/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438292243669581824)} 0 28
[junit4:junit4] 2> 602430 T1067 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438292243669581824)} 0 72
[junit4:junit4] 2> 602433 T1096 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 86
[junit4:junit4] 2> 602458 T1046 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438292243759759360)]} 0 13
[junit4:junit4] 2> 602541 T1119 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438292243815333888)]} 0 7
[junit4:junit4] 2> 602564 T1083 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438292243815333888)]} 0 14
[junit4:junit4] 2> 602566 T1098 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438292243815333888)]} 0 20
[junit4:junit4] 2> 602569 T1068 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438292243815333888)]} 0 72
[junit4:junit4] 2> 602576 T1048 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438292243895025664)]} 0 3
[junit4:junit4] 2> 602611 T1099 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438292243919142912)]} 0 3
[junit4:junit4] 2> 602613 T1084 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438292243919142912)]} 0 9
[junit4:junit4] 2> 602620 T1115 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438292243919142912)]} 0 3
[junit4:junit4] 2> 602622 T1063 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49436/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438292243919142912)]} 0 25
[junit4:junit4] 2> 602624 T1120 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 43
[junit4:junit4] 2> 602635 T1043 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438292243956891648)]} 0 2
[junit4:junit4] 2> 602647 T1064 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49425/collection1/]
[junit4:junit4] 2> 602648 T1064 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49425/collection1/ against:[http://127.0.0.1:49425/collection1/] result:true
[junit4:junit4] 2> 602649 T1064 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49429/collection1/ against:[http://127.0.0.1:49425/collection1/] result:false
[junit4:junit4] 2> 602649 T1064 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49436/collection1/ against:[http://127.0.0.1:49425/collection1/] result:false
[junit4:junit4] 2> 602659 T1100 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&wt=javabin&version=2} {add=[2 (1438292243974717440)]} 0 3
[junit4:junit4] 2> 602665 T1116 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&wt=javabin&version=2} {add=[2 (1438292243974717440)]} 0 3
[junit4:junit4] 2> 602667 T1064 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49425/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&wt=javabin&version=2} {add=[2 (1438292243974717440)]} 0 21
[junit4:junit4] 2> 602669 T1079 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&wt=javabin&version=2} {add=[2]} 0 27
[junit4:junit4] 2> 602697 T1044 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438292244020854784)]} 0 3
[junit4:junit4] 2> 602704 T1066 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49429/collection1/, http://127.0.0.1:49425/collection1/]
[junit4:junit4] 2> 602705 T1066 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49425/collection1/ against:[http://127.0.0.1:49429/collection1/, http://127.0.0.1:49425/collection1/] result:true
[junit4:junit4] 2> 602706 T1066 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49429/collection1/ against:[http://127.0.0.1:49429/collection1/, http://127.0.0.1:49425/collection1/] result:true
[junit4:junit4] 2> 602707 T1066 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49436/collection1/ against:[http://127.0.0.1:49429/collection1/, http://127.0.0.1:49425/collection1/] result:false
[junit4:junit4] 2> 602717 T1117 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49422/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&wt=javabin&version=2} {add=[3 (1438292244035534848)]} 0 2
[junit4:junit4] 2> 602720 T1066 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49425/collection1/&test.distrib.skip.servers=http://127.0.0.1:49429/collection1/&wt=javabin&version=2} {add=[3 (1438292244035534848)]} 0 17
[junit4:junit4] 2> 602735 T1045 C269 P49418 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 602840 T1045 C269 P49418 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 602840 T1045 C269 P49418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 602849 T1045 C269 P49418 oass.SolrIndexSearcher.<init> Opening Searcher@34d48119 main
[junit4:junit4] 2> 602851 T1045 C269 P49418 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 602852 T1056 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@34d48119 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 602853 T1045 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 118
[junit4:junit4] 2> 602858 T1068 C268 P49422 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49422/collection1/, StdNode: http://127.0.0.1:49425/collection1/, StdNode: http://127.0.0.1:49429/collection1/, StdNode: http://127.0.0.1:49436/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 602867 T1065 C268 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 602868 T1101 C270 P49429 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 602885 T1080 C271 P49425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 602905 T1118 C267 P49436 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 602996 T1065 C268 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 603000 T1065 C268 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 603015 T1065 C268 P49422 oass.SolrIndexSearcher.<init> Opening Searcher@7604b710 main
[junit4:junit4] 2> 603018 T1074 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7604b710 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 603031 T1065 C268 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603032 T1065 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 165
[junit4:junit4] 2> 603076 T1101 C270 P49429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 603077 T1101 C270 P49429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 603090 T1101 C270 P49429 oass.SolrIndexSearcher.<init> Opening Searcher@35333cb6 main
[junit4:junit4] 2> 603091 T1101 C270 P49429 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603093 T1109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35333cb6 main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):c3)}
[junit4:junit4] 2> 603096 T1080 C271 P49425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 603097 T1080 C271 P49425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 603106 T1080 C271 P49425 oass.SolrIndexSearcher.<init> Opening Searcher@538582e6 main
[junit4:junit4] 2> 603115 T1080 C271 P49425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603099 T1101 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 231
[junit4:junit4] 2> 603097 T1118 C267 P49436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 603117 T1118 C267 P49436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 603115 T1090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@538582e6 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):c2)}
[junit4:junit4] 2> 603119 T1080 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 234
[junit4:junit4] 2> 603123 T1118 C267 P49436 oass.SolrIndexSearcher.<init> Opening Searcher@11a478cd main
[junit4:junit4] 2> 603125 T1118 C267 P49436 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603127 T1128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11a478cd main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 603128 T1118 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 223
[junit4:junit4] 2> 603132 T1068 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 274
[junit4:junit4] 2> 603134 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 603136 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 603138 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 603142 T1064 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 603147 T1082 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=1
[junit4:junit4] 2> 603151 T1096 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=3 status=0 QTime=0
[junit4:junit4] 2> 603156 T1115 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 603169 T1030 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 603216 T1099 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 603220 T1099 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 603247 T1066 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 603248 T1066 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49422/collection1/
[junit4:junit4] 2> 603249 T1066 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49422 START replicas=[http://127.0.0.1:49425/collection1/, http://127.0.0.1:49429/collection1/, http://127.0.0.1:49436/collection1/] nUpdates=100
[junit4:junit4] 2> 603259 T1083 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 603262 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Received 3 versions from 127.0.0.1:49425/collection1/
[junit4:junit4] 2> 603263 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Our versions are newer. ourLowThreshold=1438292243669581824 otherHigh=1438292243919142912
[junit4:junit4] 2> 603265 T1116 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 603266 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Received 5 versions from 127.0.0.1:49436/collection1/
[junit4:junit4] 2> 603267 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Our versions are newer. ourLowThreshold=1438292243669581824 otherHigh=1438292243974717440
[junit4:junit4] 2> 603269 T1100 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 603271 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Received 4 versions from 127.0.0.1:49429/collection1/
[junit4:junit4] 2> 603272 T1066 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49422 Our versions are newer. ourLowThreshold=1438292243669581824 otherHigh=1438292243974717440
[junit4:junit4] 2> 603272 T1066 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49422 DONE. sync succeeded
[junit4:junit4] 2> 603273 T1066 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 603273 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: try and ask http://127.0.0.1:49425/collection1/ to sync
[junit4:junit4] 2> 603275 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: try and ask http://127.0.0.1:49429/collection1/ to sync
[junit4:junit4] 2> 603278 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: try and ask http://127.0.0.1:49436/collection1/ to sync
[junit4:junit4] 2> 603293 T1117 C267 P49436 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49436 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 603297 T1101 C270 P49429 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49429 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 603308 T1067 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 603310 T1084 C271 P49425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49425 START replicas=[http://127.0.0.1:49422/collection1/] nUpdates=100
[junit4:junit4] 2> 603311 T1117 C267 P49436 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49436 Received 5 versions from 127.0.0.1:49422/collection1/
[junit4:junit4] 2> 603314 T1117 C267 P49436 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49436 Our versions are newer. ourLowThreshold=1438292243669581824 otherHigh=1438292243974717440
[junit4:junit4] 2> 603315 T1117 C267 P49436 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49436 DONE. sync succeeded
[junit4:junit4] 2> 603315 T1117 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49422/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=22
[junit4:junit4] 2> 603318 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: sync completed with http://127.0.0.1:49436/collection1/
[junit4:junit4] 2> 603324 T1065 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 603328 T1101 C270 P49429 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49429 Received 5 versions from 127.0.0.1:49422/collection1/
[junit4:junit4] 2> 603329 T1101 C270 P49429 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:49429 Requesting updates from 127.0.0.1:49422/collection1/n=1 versions=[1438292244035534848]
[junit4:junit4] 2> 603335 T1068 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438292244035534848&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 603337 T1063 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 603339 T1084 C271 P49425 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49425 Received 5 versions from 127.0.0.1:49422/collection1/
[junit4:junit4] 2> 603340 T1084 C271 P49425 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:49425 Requesting updates from 127.0.0.1:49422/collection1/n=2 versions=[1438292244035534848, 1438292243974717440]
[junit4:junit4] 2> 603345 T1064 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438292244035534848,1438292243974717440&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 603368 T1101 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] {add=[3 (1438292244035534848)]} 0 19
[junit4:junit4] 2> 603369 T1084 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438292243974717440), 3 (1438292244035534848)]} 0 22
[junit4:junit4] 2> 603370 T1084 C271 P49425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49425 DONE. sync succeeded
[junit4:junit4] 2> 603370 T1084 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49422/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=60
[junit4:junit4] 2> 603369 T1101 C270 P49429 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49429 DONE. sync succeeded
[junit4:junit4] 2> 603372 T1101 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49422/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=75
[junit4:junit4] 2> 603377 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: sync completed with http://127.0.0.1:49425/collection1/
[junit4:junit4] 2> 603378 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:49422/collection1/: sync completed with http://127.0.0.1:49429/collection1/
[junit4:junit4] 2> 603379 T1066 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=132
[junit4:junit4] 2> 603380 T1099 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=175
[junit4:junit4] 2> 603380 T1030 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 603381 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 603386 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 603390 T1030 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 603395 T1048 C269 P49418 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 603405 T1048 C269 P49418 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 603406 T1048 C269 P49418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 603410 T1048 C269 P49418 oass.SolrIndexSearcher.<init> Opening Searcher@64ff02dc main
[junit4:junit4] 2> 603411 T1048 C269 P49418 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603414 T1056 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64ff02dc main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 603415 T1048 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 603421 T1096 C270 P49429 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49422/collection1/, StdNode: http://127.0.0.1:49425/collection1/, StdNode: http://127.0.0.1:49429/collection1/, StdNode: http://127.0.0.1:49436/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 603432 T1098 C270 P49429 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 603439 T1118 C267 P49436 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 603440 T1067 C268 P49422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 603448 T1081 C271 P49425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 603450 T1067 C268 P49422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 603451 T1067 C268 P49422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 603465 T1067 C268 P49422 oass.SolrIndexSearcher.<init> Opening Searcher@5620f476 main
[junit4:junit4] 2> 603467 T1074 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5620f476 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 603468 T1067 C268 P49422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603469 T1067 C268 P49422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 40
[junit4:junit4] 2> 603480 T1118 C267 P49436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 603481 T1118 C267 P49436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 603484 T1118 C267 P49436 oass.SolrIndexSearcher.<init> Opening Searcher@49c67ca9 main
[junit4:junit4] 2> 603484 T1118 C267 P49436 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603486 T1128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49c67ca9 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 603495 T1118 C267 P49436 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 57
[junit4:junit4] 2> 603558 T1081 C271 P49425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty2/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 603559 T1081 C271 P49425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 603558 T1098 C270 P49429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371662335565/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 603560 T1098 C270 P49429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 603566 T1098 C270 P49429 oass.SolrIndexSearcher.<init> Opening Searcher@7adf3384 main
[junit4:junit4] 2> 603569 T1109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7adf3384 main{StandardDirectoryReader(segments_6:5:nrt _0(5.0):c3 _1(5.0):c1)}
[junit4:junit4] 2> 603571 T1081 C271 P49425 oass.SolrIndexSearcher.<init> Opening Searcher@2bb7924c main
[junit4:junit4] 2> 603575 T1090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bb7924c main{StandardDirectoryReader(segments_5:5:nrt _0(5.0):c2 _1(5.0):c2)}
[junit4:junit4] 2> 603613 T1098 C270 P49429 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603614 T1098 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 182
[junit4:junit4] 2> 603616 T1081 C271 P49425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 603618 T1081 C271 P49425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 170
[junit4:junit4] 2> 603619 T1096 C270 P49429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 198
[junit4:junit4] 2> 603622 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 603630 T1065 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=2
[junit4:junit4] 2> 603638 T1080 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 603642 T1100 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 603650 T1115 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 605655 T1030 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:127.0.0.1:49422__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49422_","base_url":"http://127.0.0.1:49422","leader":"true"}
[junit4:junit4] 2> 605662 T1065 C268 P49422 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:127.0.0.1:49425__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49425_","base_url":"http://127.0.0.1:49425"}
[junit4:junit4] 2> 605668 T1080 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:127.0.0.1:49429__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49429_","base_url":"http://127.0.0.1:49429"}
[junit4:junit4] 2> 605675 T1100 C270 P49429 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:127.0.0.1:49436__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49436_","base_url":"http://127.0.0.1:49436"}
[junit4:junit4] 2> 605681 T1115 C267 P49436 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> 605688 T1082 C271 P49425 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&wt=javabin&version=2} hits=4 status=0 QTime=1
[junit4:junit4] 2> 605701 T1048 C269 P49418 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438292247168679936)]} 0 6
[junit4:junit4] 2> 605711 T1068 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49425/collection1/]
[junit4:junit4] 2> 605712 T1068 C268 P49422 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49425/collection1/ against:[http://127.0.0.1:49425/collection1/
[...truncated too long message...]
"127.0.0.1:49425__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49425_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49425"},
[junit4:junit4] 1> "127.0.0.1:49429__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49429_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49429",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "127.0.0.1:49436__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49436_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49436"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:49418__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:49418_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49418",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49436_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49418_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49422_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49429_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893262824439822-127.0.0.1:49422__collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893262824439820-127.0.0.1:49436__collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89893262824439818-127.0.0.1:49429__collection1-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49429_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49429"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89893262824439811-127.0.0.1:49418__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49418_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49418"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89893262824439822-127.0.0.1:49422_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893262824439820-127.0.0.1:49436_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893262824439818-127.0.0.1:49429_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89893262824439811-127.0.0.1:49418_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89893262824439811-127.0.0.1:49418_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=3953250FFE0FF89E -Dtests.slow=true -Dtests.locale=zh_CN -Dtests.timezone=Etc/GMT-12 -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 136s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49422/collection1lastClient and got 261 from http://127.0.0.1:49429/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([3953250FFE0FF89E:B8B5AB17895098A2]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 701749 T1030 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 136115 T1029 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 701868 T1124 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 701869 T1124 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=zh_CN, timezone=Etc/GMT-12
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=3,free=289391600,total=520093696
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SolrIndexConfigTest, ClusterStateTest, DirectUpdateHandlerTest, ChaosMonkeyNothingIsSafeTest, DocumentAnalysisRequestHandlerTest, OverseerTest, LukeRequestHandlerTest, TermVectorComponentTest, TestCloudManagedSchema, OverseerCollectionProcessorTest, MinimalSchemaTest, OpenCloseCoreStressTest, DirectSolrSpellCheckerTest, SimpleFacetsTest, TestDynamicFieldResource, PluginInfoTest, AlternateDirectoryTest, SoftAutoCommitTest, NumericFieldsTest, UpdateParamsTest, ShardRoutingCustomTest, BadCopyFieldTest, RequestHandlersTest, TestUtils, TestRecovery, SOLR749Test, XmlUpdateRequestHandlerTest, LeaderElectionIntegrationTest, TestCodecSupport, TestFoldingMultitermQuery, SyncSliceTest]
[junit4:junit4] Completed in 136.54s, 1 test, 1 failure <<< FAILURES!
[...truncated 822 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 118 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 576 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/576/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:49521/collection1lastClient and got 251 from http://127.0.0.1:49528/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49521/collection1lastClient and got 251 from http://127.0.0.1:49528/collection1
at __randomizedtesting.SeedInfo.seed([5B81A085A9908704:DA672E9DDECFE738]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9381 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 894968 T1392 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 894984 T1392 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371637078751
[junit4:junit4] 2> 894988 T1392 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 894990 T1393 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 895091 T1392 oasc.ZkTestServer.run start zk server on port:49514
[junit4:junit4] 2> 895096 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 895110 T1399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74c78043 name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 895115 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 895116 T1392 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 895128 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 895133 T1401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3279bfcb name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 895134 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 895134 T1392 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 895147 T1392 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 895164 T1392 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 895172 T1392 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 895181 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 895183 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 895198 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 895200 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 895210 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 895211 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 895221 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 895222 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 895240 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 895241 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 895251 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 895252 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 895261 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 895262 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 895272 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 895274 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 895294 T1392 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 895295 T1392 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 895902 T1392 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 895932 T1392 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49517
[junit4:junit4] 2> 895934 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 895935 T1392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 895936 T1392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072
[junit4:junit4] 2> 895937 T1392 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/solr.xml
[junit4:junit4] 2> 895938 T1392 oasc.CoreContainer.<init> New CoreContainer 1816946244
[junit4:junit4] 2> 895939 T1392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/'
[junit4:junit4] 2> 895941 T1392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/'
[junit4:junit4] 2> 896146 T1392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 896146 T1392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 896147 T1392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 896147 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 896151 T1392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 896152 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 896156 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 896157 T1392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 896158 T1392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 896158 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 896188 T1392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 896189 T1392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49514/solr
[junit4:junit4] 2> 896189 T1392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 896192 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 896199 T1412 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3028b65a name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 896200 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 896205 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 896238 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 896244 T1414 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f92343b name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 896244 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 896249 T1392 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 896260 T1392 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 896270 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 896275 T1392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49517_
[junit4:junit4] 2> 896278 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49517_
[junit4:junit4] 2> 896291 T1392 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 896321 T1392 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 896331 T1392 oasc.Overseer.start Overseer (id=89891607594270723-127.0.0.1:49517_-n_0000000000) starting
[junit4:junit4] 2> 896342 T1392 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 896355 T1416 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 896359 T1392 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 896370 T1392 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 896385 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 896406 T1417 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 896406 T1417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 896407 T1415 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 896415 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 896418 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49517_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49517"}
[junit4:junit4] 2> 896418 T1415 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 896419 T1415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 896432 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 897412 T1417 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/collection1
[junit4:junit4] 2> 897412 T1417 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 897415 T1417 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 897415 T1417 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 897420 T1417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/collection1/'
[junit4:junit4] 2> 897421 T1417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 897423 T1417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/collection1/lib/README' to classloader
[junit4:junit4] 2> 897525 T1417 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 897652 T1417 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 897657 T1417 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 897672 T1417 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 898886 T1417 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 898901 T1417 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 898907 T1417 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 898956 T1417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 898965 T1417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 898975 T1417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 898979 T1417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 898979 T1417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 898979 T1417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 898982 T1417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 898983 T1417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 898983 T1417 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 899060 T1417 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371637079072/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/
[junit4:junit4] 2> 899060 T1417 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@413c6f94
[junit4:junit4] 2> 899061 T1417 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 899063 T1417 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data
[junit4:junit4] 2> 899064 T1417 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index/
[junit4:junit4] 2> 899065 T1417 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 899066 T1417 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index
[junit4:junit4] 2> 899074 T1417 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 899075 T1417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 899079 T1417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 899079 T1417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 899080 T1417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 899081 T1417 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 899082 T1417 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 899082 T1417 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 899083 T1417 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 899084 T1417 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 899084 T1417 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 899103 T1417 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 899116 T1417 oass.SolrIndexSearcher.<init> Opening Searcher@3df8857d main
[junit4:junit4] 2> 899117 T1417 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 899118 T1417 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 899122 T1418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3df8857d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 899132 T1417 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 899132 T1417 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49517 collection:control_collection shard:shard1
[junit4:junit4] 2> 899138 T1417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 899161 T1417 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 899169 T1417 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 899169 T1417 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 899169 T1417 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49517/collection1/
[junit4:junit4] 2> 899170 T1417 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 899170 T1417 oasc.SyncStrategy.syncToMe http://127.0.0.1:49517/collection1/ has no replicas
[junit4:junit4] 2> 899171 T1417 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49517/collection1/
[junit4:junit4] 2> 899171 T1417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 899474 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 899505 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 899563 T1417 oasc.ZkController.register We are http://127.0.0.1:49517/collection1/ and leader is http://127.0.0.1:49517/collection1/
[junit4:junit4] 2> 899564 T1417 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49517
[junit4:junit4] 2> 899564 T1417 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 899564 T1417 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 899565 T1417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 899569 T1417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 899572 T1392 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 899573 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 899574 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 899616 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 899619 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 899625 T1421 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5133a873 name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 899626 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 899633 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 899642 T1392 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 900369 T1392 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 900377 T1392 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49521
[junit4:junit4] 2> 900381 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 900383 T1392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 900384 T1392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407
[junit4:junit4] 2> 900385 T1392 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/solr.xml
[junit4:junit4] 2> 900385 T1392 oasc.CoreContainer.<init> New CoreContainer 1442826427
[junit4:junit4] 2> 900387 T1392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/'
[junit4:junit4] 2> 900388 T1392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/'
[junit4:junit4] 2> 900664 T1392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 900665 T1392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 900665 T1392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 900666 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 900667 T1392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 900667 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 900668 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 900668 T1392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 900669 T1392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 900670 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 900760 T1392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 900761 T1392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49514/solr
[junit4:junit4] 2> 900761 T1392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 900772 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 900793 T1432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e88bea2 name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 900799 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 900807 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 900863 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 900869 T1434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65331ecd name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 900875 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 900921 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 901030 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 901032 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49517_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49517"}
[junit4:junit4] 2> 901047 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 901047 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 901047 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 901935 T1392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49521_
[junit4:junit4] 2> 901939 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49521_
[junit4:junit4] 2> 901955 T1421 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 901955 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 901956 T1434 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 901957 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 901957 T1414 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 901958 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 901992 T1435 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 901992 T1435 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 902572 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 902579 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49521_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49521"}
[junit4:junit4] 2> 902581 T1415 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 902582 T1415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 902595 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 902596 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 902596 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 902998 T1435 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/collection1
[junit4:junit4] 2> 902998 T1435 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 903001 T1435 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 903002 T1435 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 903009 T1435 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/collection1/'
[junit4:junit4] 2> 903012 T1435 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 903014 T1435 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/collection1/lib/README' to classloader
[junit4:junit4] 2> 903172 T1435 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 903291 T1435 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 903299 T1435 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 903306 T1435 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 904552 T1435 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 904575 T1435 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 904580 T1435 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 904636 T1435 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 904644 T1435 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 904653 T1435 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 904657 T1435 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 904658 T1435 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 904658 T1435 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 904662 T1435 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 904663 T1435 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 904663 T1435 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 904664 T1435 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371637083407/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/
[junit4:junit4] 2> 904664 T1435 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@413c6f94
[junit4:junit4] 2> 904665 T1435 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 904667 T1435 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1
[junit4:junit4] 2> 904668 T1435 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index/
[junit4:junit4] 2> 904668 T1435 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 904669 T1435 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index
[junit4:junit4] 2> 904677 T1435 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 904677 T1435 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 904682 T1435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 904683 T1435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 904684 T1435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 904685 T1435 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 904686 T1435 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 904686 T1435 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 904687 T1435 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 904688 T1435 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 904689 T1435 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 904713 T1435 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 904727 T1435 oass.SolrIndexSearcher.<init> Opening Searcher@192407b6 main
[junit4:junit4] 2> 904729 T1435 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 904730 T1435 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 904736 T1436 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@192407b6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 904740 T1435 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 904740 T1435 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49521 collection:collection1 shard:shard1
[junit4:junit4] 2> 904743 T1435 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 904773 T1435 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 904780 T1435 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 904781 T1435 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 904781 T1435 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 904781 T1435 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 904782 T1435 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/ has no replicas
[junit4:junit4] 2> 904782 T1435 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 904782 T1435 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 905629 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 905655 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 905655 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 905656 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 905689 T1435 oasc.ZkController.register We are http://127.0.0.1:49521/collection1/ and leader is http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 905689 T1435 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49521
[junit4:junit4] 2> 905689 T1435 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 905690 T1435 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 905690 T1435 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 905696 T1435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 905705 T1392 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 905706 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 905707 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 906363 T1392 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 906377 T1392 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49524
[junit4:junit4] 2> 906379 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 906379 T1392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 906380 T1392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506
[junit4:junit4] 2> 906381 T1392 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/solr.xml
[junit4:junit4] 2> 906381 T1392 oasc.CoreContainer.<init> New CoreContainer 1817454385
[junit4:junit4] 2> 906382 T1392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/'
[junit4:junit4] 2> 906383 T1392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/'
[junit4:junit4] 2> 906612 T1392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 906613 T1392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 906614 T1392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 906614 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 906615 T1392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 906615 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 906616 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 906616 T1392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 906617 T1392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 906618 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 906647 T1392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 906648 T1392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49514/solr
[junit4:junit4] 2> 906648 T1392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 906654 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 906664 T1448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@274f60d4 name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 906666 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 906671 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 906697 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 906703 T1450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46b4c0af name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 906703 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 906721 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 907181 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 907183 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49521_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49521"}
[junit4:junit4] 2> 907195 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 907195 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 907195 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 907195 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 907734 T1392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49524_
[junit4:junit4] 2> 907737 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49524_
[junit4:junit4] 2> 907743 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 907746 T1414 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 907747 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 907755 T1421 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 907755 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 907759 T1434 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 907760 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 907766 T1450 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 907772 T1451 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 907774 T1451 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 908717 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 908787 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49524_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49524"}
[junit4:junit4] 2> 908787 T1415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 908787 T1415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 908889 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 908893 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 908894 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 908895 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 909779 T1451 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/collection1
[junit4:junit4] 2> 909780 T1451 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 909782 T1451 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 909782 T1451 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 909789 T1451 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/collection1/'
[junit4:junit4] 2> 909791 T1451 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 909792 T1451 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/collection1/lib/README' to classloader
[junit4:junit4] 2> 909899 T1451 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 910002 T1451 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 910006 T1451 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 910014 T1451 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 911151 T1451 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 911174 T1451 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 911182 T1451 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 911231 T1451 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 911243 T1451 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 911252 T1451 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 911256 T1451 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 911257 T1451 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 911258 T1451 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 911261 T1451 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 911261 T1451 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 911262 T1451 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 911262 T1451 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371637089506/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/
[junit4:junit4] 2> 911263 T1451 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@413c6f94
[junit4:junit4] 2> 911264 T1451 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 911265 T1451 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2
[junit4:junit4] 2> 911266 T1451 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index/
[junit4:junit4] 2> 911267 T1451 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 911268 T1451 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index
[junit4:junit4] 2> 911294 T1451 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 911303 T1451 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 911311 T1451 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 911312 T1451 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 911312 T1451 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 911314 T1451 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 911315 T1451 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 911315 T1451 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 911316 T1451 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 911317 T1451 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 911318 T1451 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 911349 T1451 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 911363 T1451 oass.SolrIndexSearcher.<init> Opening Searcher@193b3bba main
[junit4:junit4] 2> 911365 T1451 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 911365 T1451 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 911380 T1452 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@193b3bba main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 911384 T1451 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 911384 T1451 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49524 collection:collection1 shard:shard1
[junit4:junit4] 2> 911401 T1451 oasc.ZkController.register We are http://127.0.0.1:49524/collection1/ and leader is http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 911402 T1451 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49524
[junit4:junit4] 2> 911402 T1451 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 911402 T1451 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C299 name=collection1 org.apache.solr.core.SolrCore@6fc03089 url=http://127.0.0.1:49524/collection1 node=127.0.0.1:49524_ C299_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49524_, base_url=http://127.0.0.1:49524}
[junit4:junit4] 2> 911404 T1453 C299 P49524 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 911408 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 911409 T1453 C299 P49524 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 911409 T1453 C299 P49524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 911411 T1451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 911418 T1392 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 911419 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 911420 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 911437 T1453 C299 P49524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 911515 T1428 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 911925 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 911929 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49524_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49524"}
[junit4:junit4] 2> 911947 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 911948 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 911949 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 911950 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 912307 T1392 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 912322 T1392 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49528
[junit4:junit4] 2> 912323 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 912324 T1392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 912325 T1392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244
[junit4:junit4] 2> 912325 T1392 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/solr.xml
[junit4:junit4] 2> 912326 T1392 oasc.CoreContainer.<init> New CoreContainer 137883920
[junit4:junit4] 2> 912327 T1392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/'
[junit4:junit4] 2> 912328 T1392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/'
[junit4:junit4] 2> 912517 T1428 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 912518 T1428 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:49524_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 912568 T1392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 912568 T1392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 912569 T1392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 912569 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 912570 T1392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 912570 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 912571 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 912572 T1392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 912572 T1392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 912573 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 912606 T1392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 912607 T1392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49514/solr
[junit4:junit4] 2> 912608 T1392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 912612 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 912617 T1465 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51ec4ecf name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 912618 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 912623 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 912651 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 912656 T1467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7831b651 name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 912657 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 912678 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 913691 T1392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49528_
[junit4:junit4] 2> 913695 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49528_
[junit4:junit4] 2> 913704 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 913710 T1421 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 913710 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 913711 T1434 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 913711 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 913712 T1414 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 913712 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 913715 T1467 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 913721 T1450 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 913736 T1468 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 913736 T1468 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C299_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49524_, base_url=http://127.0.0.1:49524}
[junit4:junit4] 2> 914522 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49521/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 914523 T1453 C299 P49524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49524 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 914524 T1453 C299 P49524 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 914532 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 914535 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 914536 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 914537 T1453 C299 P49524 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 914537 T1453 C299 P49524 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49521/collection1/. core=collection1
[junit4:junit4] 2> 914540 T1453 C299 P49524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C300 name=collection1 org.apache.solr.core.SolrCore@3dc607c9 url=http://127.0.0.1:49521/collection1 node=127.0.0.1:49521_ C300_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49521_, base_url=http://127.0.0.1:49521, leader=true}
[junit4:junit4] 2> 914558 T1428 C300 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=9
[junit4:junit4] 2> 914578 T1426 C300 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 914586 T1426 C300 P49521 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 914587 T1426 C300 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 914592 T1426 C300 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 914593 T1426 C300 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 914595 T1426 C300 P49521 oass.SolrIndexSearcher.<init> Opening Searcher@ddd2c76 realtime
[junit4:junit4] 2> 914596 T1426 C300 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 914597 T1426 C300 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 914607 T1453 C299 P49524 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 914608 T1453 C299 P49524 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 914615 T1429 C300 P49521 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 914616 T1429 C300 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=4
[junit4:junit4] 2> 914618 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 914618 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 914618 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 914623 T1430 C300 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2
[junit4:junit4] 2> 914627 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 914629 T1453 C299 P49524 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391
[junit4:junit4] 2> 914631 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d36eae6 fullCopy=false
[junit4:junit4] 2> 914636 T1428 C300 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 914639 T1453 C299 P49524 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 914643 T1453 C299 P49524 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 914644 T1453 C299 P49524 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 914651 T1453 C299 P49524 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 914654 T1453 C299 P49524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 914655 T1453 C299 P49524 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 914659 T1453 C299 P49524 oass.SolrIndexSearcher.<init> Opening Searcher@34b30 main
[junit4:junit4] 2> 914661 T1452 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@34b30 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 914662 T1453 C299 P49524 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391;done=true>>]
[junit4:junit4] 2> 914662 T1453 C299 P49524 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391
[junit4:junit4] 2> 914662 T1453 C299 P49524 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index.20130619141818391
[junit4:junit4] 2> 914664 T1453 C299 P49524 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 914664 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 914664 T1453 C299 P49524 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 914665 T1453 C299 P49524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 914670 T1453 C299 P49524 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 914988 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 914995 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49528"}
[junit4:junit4] 2> 914996 T1415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 914997 T1415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 915017 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49524_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49524"}
[junit4:junit4] 2> 915034 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 915035 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 915043 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 915043 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 915044 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 915744 T1468 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/collection1
[junit4:junit4] 2> 915744 T1468 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 915746 T1468 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 915746 T1468 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 915751 T1468 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/collection1/'
[junit4:junit4] 2> 915752 T1468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 915753 T1468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/collection1/lib/README' to classloader
[junit4:junit4] 2> 915890 T1468 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 916023 T1468 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 916028 T1468 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 916035 T1468 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 917098 T1468 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 917115 T1468 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 917122 T1468 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 917172 T1468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 917181 T1468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 917190 T1468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 917194 T1468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 917197 T1468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 917197 T1468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 917200 T1468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 917201 T1468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 917203 T1468 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 917205 T1468 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371637095244/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/
[junit4:junit4] 2> 917208 T1468 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@413c6f94
[junit4:junit4] 2> 917212 T1468 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 917213 T1468 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3
[junit4:junit4] 2> 917214 T1468 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index/
[junit4:junit4] 2> 917215 T1468 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 917216 T1468 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index
[junit4:junit4] 2> 917224 T1468 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 917225 T1468 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 917230 T1468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 917231 T1468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 917232 T1468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 917233 T1468 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 917235 T1468 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 917235 T1468 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 917242 T1468 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 917243 T1468 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 917243 T1468 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 917272 T1468 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 917285 T1468 oass.SolrIndexSearcher.<init> Opening Searcher@578baa79 main
[junit4:junit4] 2> 917287 T1468 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 917288 T1468 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 917304 T1471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@578baa79 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 917308 T1468 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 917308 T1468 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49528 collection:collection1 shard:shard1
[junit4:junit4] 2> 917325 T1468 oasc.ZkController.register We are http://127.0.0.1:49528/collection1/ and leader is http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 917325 T1468 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49528
[junit4:junit4] 2> 917325 T1468 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 917326 T1468 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C301 name=collection1 org.apache.solr.core.SolrCore@7a84a043 url=http://127.0.0.1:49528/collection1 node=127.0.0.1:49528_ C301_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49528_, base_url=http://127.0.0.1:49528}
[junit4:junit4] 2> 917327 T1472 C301 P49528 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 917328 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 917328 T1472 C301 P49528 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 917329 T1472 C301 P49528 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 917332 T1472 C301 P49528 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 917337 T1468 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 917375 T1425 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 917376 T1392 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 917377 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 917378 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 918026 T1392 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 918084 T1392 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49535
[junit4:junit4] 2> 918085 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 918086 T1392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 918086 T1392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164
[junit4:junit4] 2> 918087 T1392 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/solr.xml
[junit4:junit4] 2> 918087 T1392 oasc.CoreContainer.<init> New CoreContainer 1865122644
[junit4:junit4] 2> 918088 T1392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/'
[junit4:junit4] 2> 918089 T1392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/'
[junit4:junit4] 2> 918106 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 918108 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49528"}
[junit4:junit4] 2> 918125 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 918125 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 918126 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 918126 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 918126 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 918326 T1392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 918327 T1392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 918327 T1392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 918328 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 918329 T1392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 918329 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 918330 T1392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 918330 T1392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 918335 T1392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 918336 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 918377 T1425 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 918378 T1425 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:49528_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 918405 T1392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 918406 T1392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49514/solr
[junit4:junit4] 2> 918407 T1392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 918411 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 918416 T1484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f77f1a3 name:ZooKeeperConnection Watcher:127.0.0.1:49514 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 918417 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 918432 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 918459 T1392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 918471 T1486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40e6aa39 name:ZooKeeperConnection Watcher:127.0.0.1:49514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 918473 T1392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 918493 T1392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 919507 T1392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49535_
[junit4:junit4] 2> 919511 T1392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49535_
[junit4:junit4] 2> 919530 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 919541 T1421 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919542 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919562 T1450 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919563 T1434 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919563 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919570 T1486 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919574 T1414 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919577 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919579 T1467 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 919579 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919580 T1487 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 919581 T1487 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 919654 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 919656 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49535"}
[junit4:junit4] 2> 919656 T1415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 919656 T1415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 919671 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919671 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919671 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919672 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919673 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 919674 T1486 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> C301_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49528_, base_url=http://127.0.0.1:49528}
[junit4:junit4] 2> 920385 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49521/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 920386 T1472 C301 P49528 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49528 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 920386 T1472 C301 P49528 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 920387 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 920387 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 920387 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 920388 T1472 C301 P49528 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 920388 T1472 C301 P49528 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49521/collection1/. core=collection1
[junit4:junit4] 2> 920389 T1472 C301 P49528 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C302 name=collection1 org.apache.solr.core.SolrCore@3dc607c9 url=http://127.0.0.1:49521/collection1 node=127.0.0.1:49521_ C302_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49521_, base_url=http://127.0.0.1:49521, leader=true}
[junit4:junit4] 2> 920410 T1425 C302 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 920433 T1430 C302 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 920442 T1430 C302 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 920444 T1430 C302 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 920446 T1430 C302 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 920447 T1430 C302 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 920449 T1472 C301 P49528 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 920449 T1472 C301 P49528 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 920454 T1426 C302 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 920456 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 920457 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 920457 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 920463 T1427 C302 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=2
[junit4:junit4] 2> 920466 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 920470 T1472 C301 P49528 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231
[junit4:junit4] 2> 920471 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231 lockFactory=org.apache.lucene.store.NativeFSLockFactory@651f8dbe fullCopy=false
[junit4:junit4] 2> 920479 T1425 C302 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 920485 T1472 C301 P49528 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 920490 T1472 C301 P49528 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 920490 T1472 C301 P49528 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 920497 T1472 C301 P49528 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 920499 T1472 C301 P49528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 920500 T1472 C301 P49528 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 920502 T1472 C301 P49528 oass.SolrIndexSearcher.<init> Opening Searcher@3796d6d7 main
[junit4:junit4] 2> 920504 T1471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3796d6d7 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 920506 T1472 C301 P49528 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231;done=true>>]
[junit4:junit4] 2> 920507 T1472 C301 P49528 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231
[junit4:junit4] 2> 920507 T1472 C301 P49528 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index.20130619141824231
[junit4:junit4] 2> 920510 T1472 C301 P49528 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 920510 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 920511 T1472 C301 P49528 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 920512 T1472 C301 P49528 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 920516 T1472 C301 P49528 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 920589 T1487 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/collection1
[junit4:junit4] 2> 920590 T1487 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 920592 T1487 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 920593 T1487 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 920597 T1487 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/collection1/'
[junit4:junit4] 2> 920599 T1487 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 920600 T1487 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/collection1/lib/README' to classloader
[junit4:junit4] 2> 920742 T1487 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 920842 T1487 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 920847 T1487 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 920862 T1487 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 921204 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 921211 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49528_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49528"}
[junit4:junit4] 2> 921252 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921253 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921253 T1486 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921254 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921254 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921254 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 921977 T1487 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 921993 T1487 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 921998 T1487 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 922047 T1487 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 922056 T1487 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 922066 T1487 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 922071 T1487 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 922071 T1487 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 922072 T1487 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 922075 T1487 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 922076 T1487 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 922076 T1487 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 922076 T1487 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371637101164/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/
[junit4:junit4] 2> 922077 T1487 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@413c6f94
[junit4:junit4] 2> 922078 T1487 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 922079 T1487 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4
[junit4:junit4] 2> 922081 T1487 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index/
[junit4:junit4] 2> 922082 T1487 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 922083 T1487 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index
[junit4:junit4] 2> 922094 T1487 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 922096 T1487 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 922103 T1487 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 922103 T1487 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 922104 T1487 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 922105 T1487 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 922106 T1487 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 922107 T1487 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 922111 T1487 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 922112 T1487 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 922113 T1487 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 922135 T1487 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 922148 T1487 oass.SolrIndexSearcher.<init> Opening Searcher@55303c07 main
[junit4:junit4] 2> 922149 T1487 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 922150 T1487 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 922157 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55303c07 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 922162 T1487 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 922162 T1487 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49535 collection:collection1 shard:shard1
[junit4:junit4] 2> 922172 T1487 oasc.ZkController.register We are http://127.0.0.1:49535/collection1/ and leader is http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 922173 T1487 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49535
[junit4:junit4] 2> 922173 T1487 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 922173 T1487 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C303 name=collection1 org.apache.solr.core.SolrCore@1f091854 url=http://127.0.0.1:49535/collection1 node=127.0.0.1:49535_ C303_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49535_, base_url=http://127.0.0.1:49535}
[junit4:junit4] 2> 922175 T1491 C303 P49535 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 922175 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 922176 T1491 C303 P49535 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 922176 T1491 C303 P49535 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 922180 T1491 C303 P49535 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 922182 T1487 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922194 T1392 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 922196 T1392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 922197 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 922222 T1427 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 922223 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922226 T1392 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 922226 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 922228 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922838 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922840 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49535"}
[junit4:junit4] 2> 922856 T1486 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 922857 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 922858 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 922860 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 922861 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 922863 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923225 T1427 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 923226 T1427 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:49535_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> 923234 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 924240 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C303_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49535_, base_url=http://127.0.0.1:49535}
[junit4:junit4] 2> 925229 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49521/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 925230 T1491 C303 P49535 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49535 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 925231 T1491 C303 P49535 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 925231 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 925232 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 925232 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 925232 T1491 C303 P49535 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 925232 T1491 C303 P49535 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49521/collection1/. core=collection1
[junit4:junit4] 2> 925233 T1491 C303 P49535 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C304 name=collection1 org.apache.solr.core.SolrCore@3dc607c9 url=http://127.0.0.1:49521/collection1 node=127.0.0.1:49521_ C304_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49521_, base_url=http://127.0.0.1:49521, leader=true}
[junit4:junit4] 2> 925235 T1427 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 925246 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925274 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 925282 T1428 C304 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 925284 T1428 C304 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 925286 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 925287 T1428 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 925292 T1491 C303 P49535 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 925294 T1491 C303 P49535 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 925300 T1426 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 925301 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 925302 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 925302 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 925308 T1425 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 925310 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 925316 T1491 C303 P49535 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074
[junit4:junit4] 2> 925318 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074 lockFactory=org.apache.lucene.store.NativeFSLockFactory@191da9ca fullCopy=false
[junit4:junit4] 2> 925324 T1429 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1
[junit4:junit4] 2> 925328 T1491 C303 P49535 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 925333 T1491 C303 P49535 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 925333 T1491 C303 P49535 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 925344 T1491 C303 P49535 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 925348 T1491 C303 P49535 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 925349 T1491 C303 P49535 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 925350 T1491 C303 P49535 oass.SolrIndexSearcher.<init> Opening Searcher@2a7b2224 main
[junit4:junit4] 2> 925352 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a7b2224 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 925353 T1491 C303 P49535 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074;done=true>>]
[junit4:junit4] 2> 925353 T1491 C303 P49535 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074
[junit4:junit4] 2> 925353 T1491 C303 P49535 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index.20130619141829074
[junit4:junit4] 2> 925355 T1491 C303 P49535 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 925355 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 925355 T1491 C303 P49535 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 925356 T1491 C303 P49535 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 925359 T1491 C303 P49535 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 925891 T1415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925893 T1415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49535"}
[junit4:junit4] 2> 925907 T1486 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925910 T1450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925914 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925915 T1434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925915 T1467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925914 T1414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926251 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 926255 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C305 name=collection1 org.apache.solr.core.SolrCore@46c67d63 url=http://127.0.0.1:49517/collection1 node=127.0.0.1:49517_ C305_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49517_, base_url=http://127.0.0.1:49517, leader=true}
[junit4:junit4] 2> 926283 T1406 C305 P49517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 926291 T1406 C305 P49517 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 926292 T1406 C305 P49517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 926297 T1406 C305 P49517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 926298 T1406 C305 P49517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 926301 T1406 C305 P49517 oass.SolrIndexSearcher.<init> Opening Searcher@7656bdc2 main
[junit4:junit4] 2> 926301 T1406 C305 P49517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 926303 T1418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7656bdc2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 926304 T1406 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> ASYNC NEW_CORE C306 name=collection1 org.apache.solr.core.SolrCore@7a84a043 url=http://127.0.0.1:49528/collection1 node=127.0.0.1:49528_ C306_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49528_, base_url=http://127.0.0.1:49528}
[junit4:junit4] 2> 926348 T1459 C306 P49528 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49521/collection1/, StdNode: http://127.0.0.1:49524/collection1/, StdNode: http://127.0.0.1:49528/collection1/, StdNode: http://127.0.0.1:49535/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C307 name=collection1 org.apache.solr.core.SolrCore@6fc03089 url=http://127.0.0.1:49524/collection1 node=127.0.0.1:49524_ C307_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49524_, base_url=http://127.0.0.1:49524}
[junit4:junit4] 2> 926374 T1446 C307 P49524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 926376 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 926383 T1428 C304 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 926384 T1428 C304 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 926376 T1461 C306 P49528 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 926386 T1428 C304 P49521 oass.SolrIndexSearcher.<init> Opening Searcher@51a3ca23 main
[junit4:junit4] 2> 926387 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 926393 T1436 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51a3ca23 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> C303_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49535_, base_url=http://127.0.0.1:49535}
[junit4:junit4] 2> 926400 T1477 C303 P49535 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 926401 T1428 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 926407 T1461 C306 P49528 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 926408 T1461 C306 P49528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 926412 T1461 C306 P49528 oass.SolrIndexSearcher.<init> Opening Searcher@52ada8b9 main
[junit4:junit4] 2> 926412 T1461 C306 P49528 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 926414 T1471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52ada8b9 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 926414 T1461 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4] 2> 926418 T1477 C303 P49535 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 926419 T1477 C303 P49535 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 926420 T1446 C307 P49524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 926421 T1446 C307 P49524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 926423 T1477 C303 P49535 oass.SolrIndexSearcher.<init> Opening Searcher@7d483348 main
[junit4:junit4] 2> 926423 T1446 C307 P49524 oass.SolrIndexSearcher.<init> Opening Searcher@3b414b97 main
[junit4:junit4] 2> 926425 T1446 C307 P49524 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 926425 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d483348 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 926426 T1477 C303 P49535 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 926426 T1452 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b414b97 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 926426 T1477 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 926427 T1446 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 53
[junit4:junit4] 2> 926430 T1459 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 82
[junit4:junit4] 2> 926434 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 926447 T1426 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 926453 T1441 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 926458 T1457 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 926464 T1478 C303 P49535 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 928476 T1406 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438265756599975936)} 0 4
[junit4:junit4] 2> 928519 T1442 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438265756620947456&update.from=http://127.0.0.1:49521/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438265756620947456)} 0 3
[junit4:junit4] 2> 928523 T1457 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438265756620947456&update.from=http://127.0.0.1:49521/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438265756620947456)} 0 3
[junit4:junit4] 2> 928523 T1478 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438265756620947456&update.from=http://127.0.0.1:49521/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438265756620947456)} 0 2
[junit4:junit4] 2> 928527 T1426 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438265756620947456)} 0 35
[junit4:junit4] 2> 928530 T1441 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 48
[junit4:junit4] 2> 928540 T1408 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438265756667084800)]} 0 5
[junit4:junit4] 2> 928571 T1458 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438265756679667712)]} 0 7
[junit4:junit4] 2> 928573 T1479 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438265756679667712)]} 0 7
[junit4:junit4] 2> 928582 T1443 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438265756679667712)]} 0 7
[junit4:junit4] 2> 928585 T1425 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438265756679667712)]} 0 38
[junit4:junit4] 2> 928591 T1407 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438265756722659328)]} 0 2
[junit4:junit4] 2> 928607 T1461 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438265756734193664)]} 0 1
[junit4:junit4] 2> 928614 T1480 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438265756734193664)]} 0 3
[junit4:junit4] 2> 928615 T1444 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438265756734193664)]} 0 3
[junit4:junit4] 2> 928620 T1429 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49528/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438265756734193664)]} 0 20
[junit4:junit4] 2> 928625 T1460 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 29
[junit4:junit4] 2> 928637 T1409 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438265756771942400)]} 0 1
[junit4:junit4] 2> 928647 T1427 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49524/collection1/]
[junit4:junit4] 2> 928647 T1427 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49524/collection1/ against:[http://127.0.0.1:49524/collection1/] result:true
[junit4:junit4] 2> 928648 T1427 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49528/collection1/ against:[http://127.0.0.1:49524/collection1/] result:false
[junit4:junit4] 2> 928648 T1427 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49535/collection1/ against:[http://127.0.0.1:49524/collection1/] result:false
[junit4:junit4] 2> 928659 T1459 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[2 (1438265756784525312)]} 0 2
[junit4:junit4] 2> 928659 T1481 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[2 (1438265756784525312)]} 0 2
[junit4:junit4] 2> 928662 T1427 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49528/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[2 (1438265756784525312)]} 0 16
[junit4:junit4] 2> 928672 T1462 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[2]} 0 29
[junit4:junit4] 2> 928692 T1410 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438265756826468352)]} 0 4
[junit4:junit4] 2> 928706 T1430 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49524/collection1/]
[junit4:junit4] 2> 928707 T1430 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49524/collection1/ against:[http://127.0.0.1:49524/collection1/] result:true
[junit4:junit4] 2> 928707 T1430 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49528/collection1/ against:[http://127.0.0.1:49524/collection1/] result:false
[junit4:junit4] 2> 928708 T1430 C304 P49521 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49535/collection1/ against:[http://127.0.0.1:49524/collection1/] result:false
[junit4:junit4] 2> 928714 T1457 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[3 (1438265756847439872)]} 0 1
[junit4:junit4] 2> 928714 T1477 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49521/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[3 (1438265756847439872)]} 0 1
[junit4:junit4] 2> 928717 T1430 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49535/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&wt=javabin&version=2} {add=[3 (1438265756847439872)]} 0 11
[junit4:junit4] 2> 928719 T1482 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49524/collection1/&test.distrib.skip.servers=http://127.0.0.1:49528/collection1/&wt=javabin&version=2} {add=[3]} 0 20
[junit4:junit4] 2> 928724 T1405 C305 P49517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 928839 T1405 C305 P49517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 928840 T1405 C305 P49517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 928848 T1405 C305 P49517 oass.SolrIndexSearcher.<init> Opening Searcher@2106810e main
[junit4:junit4] 2> 928849 T1405 C305 P49517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 928851 T1418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2106810e main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 928852 T1405 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 128
[junit4:junit4] 2> 928860 T1458 C306 P49528 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49521/collection1/, StdNode: http://127.0.0.1:49524/collection1/, StdNode: http://127.0.0.1:49528/collection1/, StdNode: http://127.0.0.1:49535/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 928867 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 928867 T1461 C306 P49528 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 928871 T1445 C307 P49524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 928870 T1478 C303 P49535 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929043 T1478 C303 P49535 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 929044 T1478 C303 P49535 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 929050 T1478 C303 P49535 oass.SolrIndexSearcher.<init> Opening Searcher@60ff371 main
[junit4:junit4] 2> 929051 T1478 C303 P49535 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929053 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60ff371 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929054 T1478 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 184
[junit4:junit4] 2> 929056 T1461 C306 P49528 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 929074 T1461 C306 P49528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 929085 T1461 C306 P49528 oass.SolrIndexSearcher.<init> Opening Searcher@123fb1a8 main
[junit4:junit4] 2> 929089 T1471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@123fb1a8 main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929095 T1428 C304 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 929097 T1428 C304 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 929099 T1461 C306 P49528 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929125 T1461 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 258
[junit4:junit4] 2> 929127 T1428 C304 P49521 oass.SolrIndexSearcher.<init> Opening Searcher@1abcd676 main
[junit4:junit4] 2> 929129 T1428 C304 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929131 T1436 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1abcd676 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929142 T1445 C307 P49524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 929144 T1445 C307 P49524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 929143 T1428 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 276
[junit4:junit4] 2> 929157 T1445 C307 P49524 oass.SolrIndexSearcher.<init> Opening Searcher@72f77757 main
[junit4:junit4] 2> 929158 T1445 C307 P49524 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929159 T1452 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72f77757 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):c2)}
[junit4:junit4] 2> 929160 T1445 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 290
[junit4:junit4] 2> 929161 T1458 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 301
[junit4:junit4] 2> 929163 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 929166 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 929169 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 929174 T1425 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 929179 T1442 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 929184 T1459 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 929189 T1480 C303 P49535 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 929190 T1392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 929241 T1462 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 929242 T1462 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 929274 T1427 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 929275 T1427 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49521/collection1/
[junit4:junit4] 2> 929276 T1427 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49521 START replicas=[http://127.0.0.1:49524/collection1/, http://127.0.0.1:49528/collection1/, http://127.0.0.1:49535/collection1/] nUpdates=100
[junit4:junit4] 2> 929299 T1443 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 929300 T1457 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 929325 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Received 5 versions from 127.0.0.1:49528/collection1/
[junit4:junit4] 2> 929325 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Our versions are newer. ourLowThreshold=1438265756620947456 otherHigh=1438265756784525312
[junit4:junit4] 2> 929326 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Received 3 versions from 127.0.0.1:49524/collection1/
[junit4:junit4] 2> 929326 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Our versions are newer. ourLowThreshold=1438265756620947456 otherHigh=1438265756734193664
[junit4:junit4] 2> 929328 T1481 C303 P49535 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 929330 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Received 5 versions from 127.0.0.1:49535/collection1/
[junit4:junit4] 2> 929331 T1427 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49521 Our versions are newer. ourLowThreshold=1438265756620947456 otherHigh=1438265756784525312
[junit4:junit4] 2> 929331 T1427 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49521 DONE. sync succeeded
[junit4:junit4] 2> 929332 T1427 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 929332 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: try and ask http://127.0.0.1:49524/collection1/ to sync
[junit4:junit4] 2> 929333 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: try and ask http://127.0.0.1:49528/collection1/ to sync
[junit4:junit4] 2> 929334 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: try and ask http://127.0.0.1:49535/collection1/ to sync
[junit4:junit4] 2> 929347 T1461 C306 P49528 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49528 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 929351 T1477 C303 P49535 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49535 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 929353 T1444 C307 P49524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49524 START replicas=[http://127.0.0.1:49521/collection1/] nUpdates=100
[junit4:junit4] 2> 929358 T1430 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 929363 T1461 C306 P49528 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49528 Received 5 versions from 127.0.0.1:49521/collection1/
[junit4:junit4] 2> 929365 T1461 C306 P49528 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49528 Our versions are newer. ourLowThreshold=1438265756620947456 otherHigh=1438265756784525312
[junit4:junit4] 2> 929366 T1461 C306 P49528 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49528 DONE. sync succeeded
[junit4:junit4] 2> 929366 T1461 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49521/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=19
[junit4:junit4] 2> 929369 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: sync completed with http://127.0.0.1:49528/collection1/
[junit4:junit4] 2> 929371 T1426 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 929372 T1426 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 929374 T1477 C303 P49535 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49535 Received 5 versions from 127.0.0.1:49521/collection1/
[junit4:junit4] 2> 929375 T1477 C303 P49535 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49535 Our versions are newer. ourLowThreshold=1438265756620947456 otherHigh=1438265756784525312
[junit4:junit4] 2> 929375 T1477 C303 P49535 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49535 DONE. sync succeeded
[junit4:junit4] 2> 929376 T1477 C303 P49535 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49521/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=25
[junit4:junit4] 2> 929377 T1444 C307 P49524 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49524 Received 5 versions from 127.0.0.1:49521/collection1/
[junit4:junit4] 2> 929377 T1444 C307 P49524 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:49524 Requesting updates from 127.0.0.1:49521/collection1/n=2 versions=[1438265756847439872, 1438265756784525312]
[junit4:junit4] 2> 929379 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: sync completed with http://127.0.0.1:49535/collection1/
[junit4:junit4] 2> 929383 T1428 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438265756847439872,1438265756784525312&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 929405 T1444 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438265756784525312), 3 (1438265756847439872)]} 0 16
[junit4:junit4] 2> 929405 T1444 C307 P49524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49524 DONE. sync succeeded
[junit4:junit4] 2> 929407 T1444 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:49521/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=53
[junit4:junit4] 2> 929409 T1427 oasc.SyncStrategy.syncToMe http://127.0.0.1:49521/collection1/: sync completed with http://127.0.0.1:49524/collection1/
[junit4:junit4] 2> 929410 T1427 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=136
[junit4:junit4] 2> 929411 T1462 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=170
[junit4:junit4] 2> 929413 T1392 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 929425 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 929429 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 929447 T1392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 929452 T1407 C305 P49517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929462 T1407 C305 P49517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 929465 T1407 C305 P49517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 929470 T1407 C305 P49517 oass.SolrIndexSearcher.<init> Opening Searcher@60bce5ae main
[junit4:junit4] 2> 929471 T1407 C305 P49517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929472 T1418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60bce5ae main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929473 T1407 C305 P49517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 929480 T1446 C307 P49524 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49521/collection1/, StdNode: http://127.0.0.1:49524/collection1/, StdNode: http://127.0.0.1:49528/collection1/, StdNode: http://127.0.0.1:49535/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 929486 T1429 C304 P49521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929503 T1429 C304 P49521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 929507 T1442 C307 P49524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929512 T1482 C303 P49535 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929511 T1460 C306 P49528 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929531 T1460 C306 P49528 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 929532 T1460 C306 P49528 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 929534 T1482 C303 P49535 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 929535 T1482 C303 P49535 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 929537 T1460 C306 P49528 oass.SolrIndexSearcher.<init> Opening Searcher@301170f main
[junit4:junit4] 2> 929537 T1482 C303 P49535 oass.SolrIndexSearcher.<init> Opening Searcher@4a2f4535 main
[junit4:junit4] 2> 929538 T1482 C303 P49535 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929539 T1471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@301170f main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929535 T1429 C304 P49521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 929540 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a2f4535 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929538 T1460 C306 P49528 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929541 T1482 C303 P49535 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 29
[junit4:junit4] 2> 929541 T1460 C306 P49528 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 30
[junit4:junit4] 2> 929545 T1429 C304 P49521 oass.SolrIndexSearcher.<init> Opening Searcher@403e3abc main
[junit4:junit4] 2> 929546 T1429 C304 P49521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929547 T1436 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@403e3abc main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 929548 T1429 C304 P49521 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 62
[junit4:junit4] 2> 929621 T1442 C307 P49524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371637078749/jetty2/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 929623 T1442 C307 P49524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 929628 T1442 C307 P49524 oass.SolrIndexSearcher.<init> Opening Searcher@c1ec70b main
[junit4:junit4] 2> 929629 T1442 C307 P49524 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929631 T1452 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c1ec70b main{StandardDirectoryReader(segments_5:5:nrt _0(5.0):c2 _1(5.0):c2)}
[junit4:junit4] 2> 929631 T1442 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 124
[junit4:junit4] 2> 929634 T1446 C307 P49524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 154
[junit4:junit4] 2> 929638 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 929643 T1430 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 929648 T1441 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 929654 T1459 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 929662 T1479 C303 P49535 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 931667 T1392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:2:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49521_","base_url":"http://127.0.0.1:49521","leader":"true"}
[junit4:junit4] 2> 931683 T1430 C304 P49521 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:3:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49524_","base_url":"http://127.0.0.1:49524"}
[junit4:junit4] 2> 931689 T1441 C307 P49524 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:4:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49528_","base_url":"http://127.0.0.1:49528"}
[junit4:junit4] 2> 931697 T1459 C306 P49528 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:5:{"shard":"shard1","state":"active","core":"collection1","c
[...truncated too long message...]
"state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:49517_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49517",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49528_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49517_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49521_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:49535_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891607594270732-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891607594270734-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891607594270730-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49528_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49528"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89891607594270723-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:49517_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49517"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89891607594270734-127.0.0.1:49521_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891607594270732-127.0.0.1:49535_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891607594270730-127.0.0.1:49528_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891607594270723-127.0.0.1:49517_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89891607594270723-127.0.0.1:49517_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=5B81A085A9908704 -Dtests.slow=true -Dtests.locale=de_AT -Dtests.timezone=NET -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 125s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:49521/collection1lastClient and got 251 from http://127.0.0.1:49528/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([5B81A085A9908704:DA672E9DDECFE738]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1019872 T1392 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 124910 T1391 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=NestedPulsing), rnd_b=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=226), intDefault=PostingsFormat(name=NestedPulsing), timestamp=PostingsFormat(name=Lucene41WithOrds), id=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=226), a_t=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=226), range_facet_si=PostingsFormat(name=Lucene41WithOrds), other_tl1=PostingsFormat(name=NestedPulsing), multiDefault=PostingsFormat(name=Lucene41WithOrds), a_si=PostingsFormat(name=NestedPulsing)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=DefaultSimilarity, locale=de_AT, timezone=NET
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=112122584,total=307625984
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestMergePolicyConfig, SchemaVersionSpecificBehaviorTest, TestSolrXml, TestSort, SolrTestCaseJ4Test, ZkCLITest, ClusterStateUpdateTest, TestPHPSerializedResponseWriter, FieldAnalysisRequestHandlerTest, ZkNodePropsTest, TestSurroundQueryParser, EchoParamsTest, TermsComponentTest, TestFunctionQuery, NoCacheHeaderTest, ShardRoutingCustomTest, TestStressReorder, RecoveryZkTest, TestFieldTypeResource, PathHierarchyTokenizerFactoryTest, OutputWriterTest, XsltUpdateRequestHandlerTest, DateFieldTest, TestTrie, SolrCoreCheckLockOnStartupTest, PreAnalyzedFieldTest, TestJmxIntegration, TestPartialUpdateDeduplication, BasicZkTest, TestGroupingSearch, TestSolrCoreProperties, SuggesterTSTTest, SolrIndexConfigTest, TestMaxScoreQueryParser, ShardRoutingTest, SuggesterWFSTTest, TestOmitPositions, StatelessScriptUpdateProcessorFactoryTest, TestZkChroot, FileBasedSpellCheckerTest, SolrRequestParserTest, LoggingHandlerTest, TestCSVLoader, TestClassNameShortening, TestSchemaResource, TestSolrQueryParserResource, TestBinaryField, TestLazyCores, TestQuerySenderListener, SolrInfoMBeanTest, TestSystemIdResolver, TestSolrQueryParserDefaultOperatorResource, TestPseudoReturnFields, TestStressRecovery, TestDocumentBuilder, TestJoin, TestSolrJ, TestMultiCoreConfBootstrap, ZkControllerTest, RegexBoostProcessorTest, MoreLikeThisHandlerTest, PeerSyncTest, TestFieldTypeCollectionResource, QueryElevationComponentTest, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestBadConfig, TestFastLRUCache, TestCoreContainer, DistanceFunctionTest, TestSchemaVersionResource, TimeZoneUtilsTest, TestPropInject, SimpleFacetsTest, TestIndexingPerformance, SOLR749Test, HighlighterConfigTest, TestSchemaSimilarityResource, UUIDFieldTest, SpatialFilterTest, TestCopyFieldCollectionResource, MinimalSchemaTest, TestSerializedLuceneMatchVersion, TestFastOutputStream, TestAtomicUpdateErrorCases, TestDefaultSearchFieldResource, SuggesterFSTTest, TestFieldCollectionResource, MBeansHandlerTest, TestValueSourceCache, SyncSliceTest]
[junit4:junit4] Completed in 125.49s, 1 test, 1 failure <<< FAILURES!
[...truncated 644 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 110 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 575 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/575/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary
Error Message:
IOException occured when talking to server at: https://127.0.0.1:51727/solr/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:51727/solr/collection1
at __randomizedtesting.SeedInfo.seed([77D8AE3379F9AC8E:25AEE02CFE16769D]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
at org.apache.solr.client.solrj.TestBatchUpdate.doIt(TestBatchUpdate.java:130)
at org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary(TestBatchUpdate.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
... 46 more
Build Log:
[...truncated 10453 lines...]
[junit4:junit4] Suite: org.apache.solr.client.solrj.TestBatchUpdate
[junit4:junit4] 1> 286604 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.SolrJettyTestBase ? Randomized ssl (true) and clientAuth (false)
[junit4:junit4] 1> 286606 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.SolrTestCaseJ4 ? ####initCore
[junit4:junit4] 1> 286608 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.SolrTestCaseJ4 ? ####initCore end
[junit4:junit4] 1> 286611 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.eclipse.jetty.server.Server ? jetty-8.1.10.v20130312
[junit4:junit4] 1> 286633 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.eclipse.jetty.util.ssl.SslContextFactory ? Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4:junit4] 1> 286642 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.eclipse.jetty.server.AbstractConnector ? Started SslSelectChannelConnector@127.0.0.1:51727
[junit4:junit4] 1> 286646 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.servlet.SolrDispatchFilter ? SolrDispatchFilter.init()
[junit4:junit4] 1> 286647 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrResourceLoader ? JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> 286648 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrResourceLoader ? using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
[junit4:junit4] 1> 286649 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CoreContainer ? looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/solr.xml
[junit4:junit4] 1> 286650 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CoreContainer ? New CoreContainer 1556624718
[junit4:junit4] 1> 286652 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CoreContainer ? Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/'
[junit4:junit4] 1> 286654 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrResourceLoader ? new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/'
[junit4:junit4] 1> 286928 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
[junit4:junit4] 1> 286931 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn
[junit4:junit4] 1> 286936 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/prop-base
[junit4:junit4] 1> 286943 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/props
[junit4:junit4] 1> 286951 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/text-base
[junit4:junit4] 1> 286958 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/tmp
[junit4:junit4] 1> 286961 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/tmp/prop-base
[junit4:junit4] 1> 286963 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/tmp/props
[junit4:junit4] 1> 286966 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/.svn/tmp/text-base
[junit4:junit4] 1> 286968 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCoreDiscoverer ? Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/core.properties, adding to cores
[junit4:junit4] 1> 286973 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting socketTimeout to: 0
[junit4:junit4] 1> 286974 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting urlScheme to: http://
[junit4:junit4] 1> 286975 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting connTimeout to: 0
[junit4:junit4] 1> 286976 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> 286976 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting corePoolSize to: 0
[junit4:junit4] 1> 286977 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> 286978 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> 286978 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting sizeOfQueue to: -1
[junit4:junit4] 1> 286979 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.handler.component.HttpShardHandlerFactory ? Setting fairnessPolicy to: false
[junit4:junit4] 1> 286980 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.client.solrj.impl.HttpClientUtil ? Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 1> 286989 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.logging.LogWatcher ? Registering Log Listener
[junit4:junit4] 1> 286996 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.CoreContainer ? Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1
[junit4:junit4] 1> 286998 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrResourceLoader ? new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/'
[junit4:junit4] 1> 287053 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrConfig ? Adding specified lib dirs to ClassLoader
[junit4:junit4] 1> 287055 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/extraction/lib).
[junit4:junit4] 1> 287056 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
[junit4:junit4] 1> 287057 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/clustering/lib).
[junit4:junit4] 1> 287058 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
[junit4:junit4] 1> 287059 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/langid/lib).
[junit4:junit4] 1> 287059 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
[junit4:junit4] 1> 287060 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/velocity/lib).
[junit4:junit4] 1> 287061 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrResourceLoader ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
[junit4:junit4] 1> 287151 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrConfig ? Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> 287324 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.Config ? Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> 287327 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.schema.IndexSchema ? Reading Solr Schema from schema.xml
[junit4:junit4] 1> 287352 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.schema.IndexSchema ? [collection1] Schema name=example
[junit4:junit4] 1> 289257 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.schema.IndexSchema ? unique key field: id
[junit4:junit4] 1> 289320 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.schema.FileExchangeRateProvider ? Reloading exchange rates from file currency.xml
[junit4:junit4] 1> 289340 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.schema.FileExchangeRateProvider ? Reloading exchange rates from file currency.xml
[junit4:junit4] 1> 289428 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> 289430 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/
[junit4:junit4] 1> 289431 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.JmxMonitoredMap ? JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ce6e2f2
[junit4:junit4] 1> 289432 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
[junit4:junit4] 1> 289433 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
[junit4:junit4] 1> 289435 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> 289437 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.CachingDirectoryFactory ? return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906
[junit4:junit4] 1> 289438 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index/
[junit4:junit4] 1> 289439 [coreLoadExecutor-332-thread-1] WARN org.apache.solr.core.SolrCore ? [collection1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index' doesn't exist. Creating new index...
[junit4:junit4] 1> 289443 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.CachingDirectoryFactory ? return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index
[junit4:junit4] 1> 289450 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_1,generation=1}
[junit4:junit4] 1> 289451 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? newest commit generation = 1
[junit4:junit4] 1> 289455 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? created json: solr.JSONResponseWriter
[junit4:junit4] 1> 289456 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? adding lazy queryResponseWriter: solr.VelocityResponseWriter
[junit4:junit4] 1> 289457 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? created velocity: solr.VelocityResponseWriter
[junit4:junit4] 1> 289458 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? created xslt: solr.XSLTResponseWriter
[junit4:junit4] 1> 289459 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.response.XSLTResponseWriter ? xsltCacheLifetimeSeconds=5
[junit4:junit4] 1> 289465 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /select: solr.SearchHandler
[junit4:junit4] 1> 289467 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /query: solr.SearchHandler
[junit4:junit4] 1> 289470 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> 289471 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /browse: solr.SearchHandler
[junit4:junit4] 1> 289473 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> 289475 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /update/json: solr.JsonUpdateRequestHandler
[junit4:junit4] 1> 289477 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /update/csv: solr.CSVRequestHandler
[junit4:junit4] 1> 289478 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
[junit4:junit4] 1> 289480 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /update/extract: solr.extraction.ExtractingRequestHandler
[junit4:junit4] 1> 289490 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.FieldAnalysisRequestHandler
[junit4:junit4] 1> 289491 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /analysis/field: solr.FieldAnalysisRequestHandler
[junit4:junit4] 1> 289493 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
[junit4:junit4] 1> 289494 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /analysis/document: solr.DocumentAnalysisRequestHandler
[junit4:junit4] 1> 289495 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /admin/: solr.admin.AdminHandlers
[junit4:junit4] 1> 289497 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /admin/ping: solr.PingRequestHandler
[junit4:junit4] 1> 289502 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /debug/dump: solr.DumpRequestHandler
[junit4:junit4] 1> 289505 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /replication: solr.ReplicationHandler
[junit4:junit4] 1> 289506 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4] 1> 289507 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /spell: solr.SearchHandler
[junit4:junit4] 1> 289508 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4] 1> 289509 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /tvrh: solr.SearchHandler
[junit4:junit4] 1> 289510 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4] 1> 289511 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /terms: solr.SearchHandler
[junit4:junit4] 1> 289512 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4] 1> 289514 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.RequestHandlers ? created /elevate: solr.SearchHandler
[junit4:junit4] 1> 289546 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.loader.XMLLoader ? xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> 289549 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.loader.XMLLoader ? xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> 289553 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.loader.XMLLoader ? xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> 289573 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@4305b01f main
[junit4:junit4] 1> 289574 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.update.UpdateHandler ? Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/tlog
[junit4:junit4] 1> 289575 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? Hard AutoCommit: if uncommited for 15000ms;
[junit4:junit4] 1> 289576 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.SolrCore ? Soft AutoCommit: disabled
[junit4:junit4] 1> 289577 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.component.SpellCheckComponent ? Initializing spell checkers
[junit4:junit4] 1> 289577 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.spelling.DirectSolrSpellChecker ? init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
[junit4:junit4] 1> 289578 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.component.SpellCheckComponent ? No queryConverter defined, using default converter
[junit4:junit4] 1> 289579 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.component.QueryElevationComponent ? Loading QueryElevation from: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/conf/elevate.xml
[junit4:junit4] 1> 289631 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.handler.ReplicationHandler ? Commits will be reserved for 10000
[junit4:junit4] 1> 289632 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener sending requests to Searcher@4305b01f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> 289635 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] webapp=null path=null params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false} hits=0 status=0 QTime=2
[junit4:junit4] 1> 289635 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener done.
[junit4:junit4] 1> 289636 [searcherExecutor-333-thread-1] INFO org.apache.solr.handler.component.SpellCheckComponent ? Loading spell index for spellchecker: default
[junit4:junit4] 1> 289636 [searcherExecutor-333-thread-1] INFO org.apache.solr.handler.component.SpellCheckComponent ? Loading spell index for spellchecker: wordbreak
[junit4:junit4] 1> 289638 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Registered new searcher Searcher@4305b01f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> 289642 [coreLoadExecutor-332-thread-1] INFO org.apache.solr.core.CoreContainer ? registering core: collection1
[junit4:junit4] 1> 289645 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.servlet.SolrDispatchFilter ? user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0
[junit4:junit4] 1> 289646 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.servlet.SolrDispatchFilter ? SolrDispatchFilter.init() done
[junit4:junit4] 1> 289647 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.SolrJettyTestBase ? Jetty Assigned Port#51727
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/./solrtest-TestBatchUpdate-1371611908906
[junit4:junit4] 1> 289661 [TEST-TestBatchUpdate.testWithBinaryBean-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Starting testWithBinaryBean
[junit4:junit4] 1> 289662 [TEST-TestBatchUpdate.testWithBinaryBean-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.client.solrj.impl.HttpClientUtil ? Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> 289787 [qtp1835258226-603] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_1,generation=1}
[junit4:junit4] 1> 289789 [qtp1835258226-603] INFO org.apache.solr.core.SolrCore ? newest commit generation = 1
[junit4:junit4] 1> 289801 [qtp1835258226-603] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@237a7a7d realtime
[junit4:junit4] 1> 289803 [qtp1835258226-603] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438239332321198080)} 0 22
[junit4:junit4] 1> 290700 [qtp1835258226-604] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1438239332496310272), 2 (1438239332499456000), 3 (1438239332499456001), 4 (1438239332499456002), 5 (1438239332500504576), 6 (1438239332500504577), 7 (1438239332501553152), 8 (1438239332501553153), 9 (1438239332502601728), 10 (1438239332502601729), ... (1000 adds)]} 0 753
[junit4:junit4] 1> 290708 [qtp1835258226-601] INFO org.apache.solr.update.UpdateHandler ? start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> 290813 [qtp1835258226-601] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_1,generation=1}
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_2,generation=2}
[junit4:junit4] 1> 290814 [qtp1835258226-601] INFO org.apache.solr.core.SolrCore ? newest commit generation = 2
[junit4:junit4] 1> 290833 [qtp1835258226-601] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@631fe9ed main
[junit4:junit4] 1> 290836 [qtp1835258226-601] INFO org.apache.solr.update.UpdateHandler ? end_commit_flush
[junit4:junit4] 1> 290848 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener sending requests to Searcher@631fe9ed main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1000)}
[junit4:junit4] 1> 290849 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener done.
[junit4:junit4] 1> 290861 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Registered new searcher Searcher@631fe9ed main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1000)}
[junit4:junit4] 1> 290863 [qtp1835258226-601] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 156
[junit4:junit4] 1> 290873 [qtp1835258226-603] INFO org.apache.solr.core.SolrCore ? [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1000 status=0 QTime=2
[junit4:junit4] 1> 290878 [TEST-TestBatchUpdate.testWithBinaryBean-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Ending testWithBinaryBean
[junit4:junit4] 1> 290928 [TEST-TestBatchUpdate.testWithBinary-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Starting testWithBinary
[junit4:junit4] 1> 290960 [qtp1835258226-605] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@75930ef4 realtime
[junit4:junit4] 1> 290962 [qtp1835258226-605] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438239333537546240)} 0 21
[junit4:junit4] 1> 291751 [qtp1835258226-604] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1438239333588926464), 2 (1438239333589975040), 3 (1438239333589975041), 4 (1438239333591023616), 5 (1438239333591023617), 6 (1438239333605703680), 7 (1438239333605703681), 8 (1438239333605703682), 9 (1438239333606752256), 10 (1438239333606752257), ... (1000 adds)]} 0 762
[junit4:junit4] 1> 305964 [commitScheduler-335-thread-1] INFO org.apache.solr.update.UpdateHandler ? start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> 306080 [commitScheduler-335-thread-1] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_2,generation=2}
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_3,generation=3}
[junit4:junit4] 1> 306081 [commitScheduler-335-thread-1] INFO org.apache.solr.core.SolrCore ? newest commit generation = 3
[junit4:junit4] 1> 306085 [commitScheduler-335-thread-1] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@67b174dc realtime
[junit4:junit4] 1> 306086 [commitScheduler-335-thread-1] INFO org.apache.solr.update.UpdateHandler ? end_commit_flush
[junit4:junit4] 1> 491935 [qtp1835258226-600] INFO org.apache.solr.update.UpdateHandler ? start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> 491940 [TEST-TestBatchUpdate.testWithBinary-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Ending testWithBinary
[junit4:junit4] 1> 491940 [qtp1835258226-600] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_3,generation=3}
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_4,generation=4}
[junit4:junit4] 1> 491943 [qtp1835258226-600] INFO org.apache.solr.core.SolrCore ? newest commit generation = 4
[junit4:junit4] 1> 491946 [qtp1835258226-600] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@1159bb0c main
[junit4:junit4] 1> 491947 [qtp1835258226-600] INFO org.apache.solr.update.UpdateHandler ? end_commit_flush
[junit4:junit4] 1> 491949 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener sending requests to Searcher@1159bb0c main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1000)}
[junit4:junit4] 1> 491950 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener done.
[junit4:junit4] 1> 491953 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Registered new searcher Searcher@1159bb0c main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1000)}
[junit4:junit4] 1> 491955 [qtp1835258226-600] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBatchUpdate -Dtests.method=testWithBinary -Dtests.seed=77D8AE3379F9AC8E -Dtests.slow=true -Dtests.locale=es_DO -Dtests.timezone=Etc/GMT-2 -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 201s | TestBatchUpdate.testWithBinary <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:51727/solr/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([77D8AE3379F9AC8E:25AEE02CFE16769D]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
[junit4:junit4] > at org.apache.solr.client.solrj.TestBatchUpdate.doIt(TestBatchUpdate.java:130)
[junit4:junit4] > at org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary(TestBatchUpdate.java:62)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
[junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
[junit4:junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
[junit4:junit4] > at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
[junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
[junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > ... 46 more
[junit4:junit4] 1> 492249 [TEST-TestBatchUpdate.testWithXml-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Starting testWithXml
[junit4:junit4] 1> 492291 [qtp1835258226-603] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@3f51ba26 realtime
[junit4:junit4] 1> 492293 [qtp1835258226-603] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438239544649449472)} 0 20
[junit4:junit4] 1> 493371 [qtp1835258226-602] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1438239544795201536), 2 (1438239544822464512), 3 (1438239544822464513), 4 (1438239544822464514), 5 (1438239544823513088), 6 (1438239544823513089), 7 (1438239544823513090), 8 (1438239544824561664), 9 (1438239544824561665), 10 (1438239544825610240), ... (1000 adds)]} 0 962
[junit4:junit4] 1> 493383 [qtp1835258226-603] INFO org.apache.solr.update.UpdateHandler ? start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> 493440 [qtp1835258226-603] INFO org.apache.solr.core.SolrCore ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_4,generation=4}
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c5d105a lockFactory=org.apache.lucene.store.NativeFSLockFactory@402d958b),segFN=segments_5,generation=5}
[junit4:junit4] 1> 493442 [qtp1835258226-603] INFO org.apache.solr.core.SolrCore ? newest commit generation = 5
[junit4:junit4] 1> 493446 [qtp1835258226-603] INFO org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@2974c545 main
[junit4:junit4] 1> 493478 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener sending requests to Searcher@2974c545 main{StandardDirectoryReader(segments_5:11:nrt _2(5.0):c1000)}
[junit4:junit4] 1> 493480 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? QuerySenderListener done.
[junit4:junit4] 1> 493488 [searcherExecutor-333-thread-1] INFO org.apache.solr.core.SolrCore ? [collection1] Registered new searcher Searcher@2974c545 main{StandardDirectoryReader(segments_5:11:nrt _2(5.0):c1000)}
[junit4:junit4] 1> 493492 [qtp1835258226-603] INFO org.apache.solr.update.UpdateHandler ? end_commit_flush
[junit4:junit4] 1> 493497 [qtp1835258226-603] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 114
[junit4:junit4] 1> 493507 [qtp1835258226-605] INFO org.apache.solr.core.SolrCore ? [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1000 status=0 QTime=3
[junit4:junit4] 1> 493513 [TEST-TestBatchUpdate.testWithXml-seed#[77D8AE3379F9AC8E]] INFO org.apache.solr.SolrTestCaseJ4 ? ###Ending testWithXml
[junit4:junit4] 1> 493554 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CoreContainer ? Shutting down CoreContainer instance=1556624718
[junit4:junit4] 1> 493556 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCore ? [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4831b871
[junit4:junit4] 1> 493594 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.update.UpdateHandler ? closing DirectUpdateHandler2{commits=4,autocommit maxTime=15000ms,autocommits=1,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3000,cumulative_deletesById=0,cumulative_deletesByQuery=3,cumulative_errors=0}
[junit4:junit4] 1> 493597 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.update.SolrCoreState ? Closing SolrCoreState
[junit4:junit4] 1> 493598 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.update.DefaultSolrCoreState ? SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 1> 493599 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.update.DefaultSolrCoreState ? closing IndexWriter with IndexWriterCloser
[junit4:junit4] 1> 493605 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.SolrCore ? [collection1] Closing main searcher on request.
[junit4:junit4] 1> 493606 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CachingDirectoryFactory ? Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 1> 493610 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CachingDirectoryFactory ? looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906;done=false>>]
[junit4:junit4] 1> 493612 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CachingDirectoryFactory ? Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906
[junit4:junit4] 1> 493619 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CachingDirectoryFactory ? looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index;done=false>>]
[junit4:junit4] 1> 493620 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.core.CachingDirectoryFactory ? Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/solrtest-TestBatchUpdate-1371611908906/index
[junit4:junit4] 1> 493621 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.eclipse.jetty.server.handler.ContextHandler ? stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 1> 493675 [SUITE-TestBatchUpdate-seed#[77D8AE3379F9AC8E]-worker] INFO org.apache.solr.SolrTestCaseJ4 ? ###deleteCore
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_DO, timezone=Etc/GMT-2
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=67028616,total=255852544
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SolrPingTest, TestJavaBinCodec, SolrExampleEmbeddedTest, TestCoreAdmin, ContentStreamTest, SolrExampleXMLTest, TestFastInputStream, SolrQueryTest, TestSolrProperties, TestXMLEscaping, NamedListTest, LargeVolumeJettyTest, SolrDocumentTest, TestEmbeddedSolrServer, MultiCoreExampleJettyTest, FieldAnalysisResponseTest, FacetFieldTest, CloudSolrServerTest, IteratorChainTest, LargeVolumeEmbeddedTest, SolrExampleBinaryTest, TestLBHttpSolrServer, SolrExampleStreamingTest, SolrExampleStreamingBinaryTest, SolrExampleJettyTest, BasicHttpSolrServerTest, LargeVolumeBinaryJettyTest, TestBatchUpdate]
[junit4:junit4] Completed in 207.24s, 3 tests, 1 error <<< FAILURES!
[...truncated 57 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:194: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 46 suites, 272 tests, 1 error
Total time: 124 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure