You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/25 20:07:40 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 595 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/595/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:51018/mwr/ms/collection1lastClient and got 265 from http://127.0.0.1:51025/mwr/ms/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51018/mwr/ms/collection1lastClient and got 265 from http://127.0.0.1:51025/mwr/ms/collection1
at __randomizedtesting.SeedInfo.seed([6E9B481AEB21F1C2:EF7DC6029C7E91FE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9621 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 2812115 T5593 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /mwr/ms
[junit4:junit4] 2> 2812126 T5593 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372182352107
[junit4:junit4] 2> 2812128 T5593 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 2812129 T5594 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 2812230 T5593 oasc.ZkTestServer.run start zk server on port:51011
[junit4:junit4] 2> 2812233 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2812245 T5600 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a91926 name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2812247 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2812248 T5593 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 2812258 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2812264 T5602 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e39d686 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2812265 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2812265 T5593 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 2812271 T5593 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 2812276 T5593 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 2812280 T5593 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 2812288 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2812289 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2812302 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 2812303 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 2812315 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2812316 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2812322 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 2812323 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 2812329 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 2812330 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 2812336 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2812336 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2812342 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2812343 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2812348 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2812349 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2812356 T5593 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2812356 T5593 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2812819 T5593 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2812833 T5593 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51014
[junit4:junit4] 2> 2812837 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2812838 T5593 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2812839 T5593 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349
[junit4:junit4] 2> 2812839 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/'
[junit4:junit4] 2> 2812884 T5593 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/solr.xml
[junit4:junit4] 2> 2812885 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/'
[junit4:junit4] 2> 2813031 T5593 oasc.CoreContainer.<init> New CoreContainer 1558876164
[junit4:junit4] 2> 2813032 T5593 oasc.CoreContainer.load Loading cores into CoreContainer 1558876164
[junit4:junit4] 2> 2813034 T5593 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2813035 T5593 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2813035 T5593 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2813036 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2813036 T5593 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2813037 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2813037 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2813038 T5593 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2813038 T5593 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2813039 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2813060 T5593 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4:junit4] 2> 2813061 T5593 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51011/solr
[junit4:junit4] 2> 2813061 T5593 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2813064 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2813069 T5613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14a8043a name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2813070 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2813075 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2813085 T5595 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f7c7110f40002, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 2813092 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2813103 T5615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4799adf0 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2813104 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2813107 T5593 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 2813117 T5593 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 2813125 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 2813130 T5593 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51014_mwr%2Fms
[junit4:junit4] 2> 2813133 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51014_mwr%2Fms
[junit4:junit4] 2> 2813142 T5593 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 2813154 T5593 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 2813165 T5593 oasc.Overseer.start Overseer (id=89927342628274179-127.0.0.1:51014_mwr%2Fms-n_0000000000) starting
[junit4:junit4] 2> 2813173 T5593 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 2813185 T5617 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 2813186 T5593 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 2813192 T5593 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 2813197 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2813210 T5618 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2813210 T5618 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2813213 T5616 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 2813218 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2813219 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51014_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51014/mwr/ms"}
[junit4:junit4] 2> 2813219 T5616 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2813219 T5616 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2813231 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2814217 T5618 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/collection1
[junit4:junit4] 2> 2814218 T5618 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 2814221 T5618 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2814222 T5618 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 2814227 T5618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/collection1/'
[junit4:junit4] 2> 2814229 T5618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2814229 T5618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/collection1/lib/README' to classloader
[junit4:junit4] 2> 2814325 T5618 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2814403 T5618 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2814406 T5618 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2814417 T5618 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2815259 T5618 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2815278 T5618 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2815283 T5618 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2815313 T5618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2815337 T5618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2815343 T5618 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2815346 T5618 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2815347 T5618 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2815347 T5618 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2815349 T5618 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2815349 T5618 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2815350 T5618 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2815350 T5618 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372182352349/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/
[junit4:junit4] 2> 2815351 T5618 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b53df31
[junit4:junit4] 2> 2815351 T5618 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2815353 T5618 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data
[junit4:junit4] 2> 2815353 T5618 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index/
[junit4:junit4] 2> 2815354 T5618 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2815355 T5618 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index
[junit4:junit4] 2> 2815364 T5618 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2815364 T5618 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2815368 T5618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2815369 T5618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2815370 T5618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2815370 T5618 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2815371 T5618 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2815371 T5618 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2815372 T5618 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2815373 T5618 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2815373 T5618 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2815387 T5618 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2815402 T5618 oass.SolrIndexSearcher.<init> Opening Searcher@25708fd9 main
[junit4:junit4] 2> 2815404 T5618 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2815404 T5618 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2815409 T5619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25708fd9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2815411 T5618 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2815411 T5618 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51014/mwr/ms collection:control_collection shard:shard1
[junit4:junit4] 2> 2815415 T5618 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2815432 T5618 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2815439 T5618 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2815439 T5618 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2815440 T5618 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51014/mwr/ms/collection1/
[junit4:junit4] 2> 2815440 T5618 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2815440 T5618 oasc.SyncStrategy.syncToMe http://127.0.0.1:51014/mwr/ms/collection1/ has no replicas
[junit4:junit4] 2> 2815441 T5618 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51014/mwr/ms/collection1/
[junit4:junit4] 2> 2815441 T5618 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2816251 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2816265 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2816281 T5618 oasc.ZkController.register We are http://127.0.0.1:51014/mwr/ms/collection1/ and leader is http://127.0.0.1:51014/mwr/ms/collection1/
[junit4:junit4] 2> 2816285 T5618 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51014/mwr/ms
[junit4:junit4] 2> 2816287 T5618 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2816287 T5618 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2816287 T5618 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2816290 T5618 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2816293 T5593 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2816293 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2816294 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2816314 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2816318 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2816322 T5622 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e9d5d01 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2816322 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2816324 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2816334 T5593 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 2816739 T5593 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2816750 T5593 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51018
[junit4:junit4] 2> 2816752 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2816752 T5593 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2816753 T5593 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315
[junit4:junit4] 2> 2816753 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/'
[junit4:junit4] 2> 2816798 T5593 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/solr.xml
[junit4:junit4] 2> 2816798 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/'
[junit4:junit4] 2> 2816916 T5593 oasc.CoreContainer.<init> New CoreContainer 1947042761
[junit4:junit4] 2> 2816916 T5593 oasc.CoreContainer.load Loading cores into CoreContainer 1947042761
[junit4:junit4] 2> 2816918 T5593 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2816919 T5593 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2816919 T5593 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2816920 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2816920 T5593 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2816920 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2816921 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2816921 T5593 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2816922 T5593 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2816923 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2816945 T5593 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4:junit4] 2> 2816946 T5593 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51011/solr
[junit4:junit4] 2> 2816946 T5593 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2816948 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2816952 T5633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@605f6c41 name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2816954 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2816957 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2816974 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2816979 T5635 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@447a6953 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2816980 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2816991 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2817778 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2817779 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51014_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51014/mwr/ms"}
[junit4:junit4] 2> 2817787 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2817791 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2817791 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2818009 T5593 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51018_mwr%2Fms
[junit4:junit4] 2> 2818017 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51018_mwr%2Fms
[junit4:junit4] 2> 2818026 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2818027 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2818030 T5635 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2818030 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2818043 T5636 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2818043 T5636 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2818047 T5615 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2818049 T5622 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2819297 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2819299 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51018_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51018/mwr/ms"}
[junit4:junit4] 2> 2819299 T5616 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 2819300 T5616 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2819317 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2819320 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2819321 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2820048 T5636 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/collection1
[junit4:junit4] 2> 2820048 T5636 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2820050 T5636 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2820050 T5636 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2820052 T5636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/collection1/'
[junit4:junit4] 2> 2820053 T5636 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2820054 T5636 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/collection1/lib/README' to classloader
[junit4:junit4] 2> 2820129 T5636 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2820234 T5636 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2820237 T5636 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2820249 T5636 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2821094 T5636 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2821107 T5636 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2821113 T5636 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2821149 T5636 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2821156 T5636 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2821163 T5636 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2821167 T5636 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2821167 T5636 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2821167 T5636 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2821170 T5636 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2821171 T5636 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2821171 T5636 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2821171 T5636 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372182356315/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/
[junit4:junit4] 2> 2821172 T5636 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b53df31
[junit4:junit4] 2> 2821172 T5636 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2821173 T5636 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1
[junit4:junit4] 2> 2821174 T5636 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index/
[junit4:junit4] 2> 2821175 T5636 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2821176 T5636 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index
[junit4:junit4] 2> 2821186 T5636 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2821187 T5636 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2821190 T5636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2821190 T5636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2821193 T5636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2821195 T5636 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2821196 T5636 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2821197 T5636 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2821197 T5636 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2821203 T5636 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2821207 T5636 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2821224 T5636 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2821239 T5636 oass.SolrIndexSearcher.<init> Opening Searcher@77103cb1 main
[junit4:junit4] 2> 2821241 T5636 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2821246 T5636 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2821251 T5637 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@77103cb1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2821254 T5636 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2821254 T5636 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51018/mwr/ms collection:collection1 shard:shard1
[junit4:junit4] 2> 2821258 T5636 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 2821285 T5636 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2821290 T5636 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2821291 T5636 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2821291 T5636 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2821291 T5636 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2821292 T5636 oasc.SyncStrategy.syncToMe http://127.0.0.1:51018/mwr/ms/collection1/ has no replicas
[junit4:junit4] 2> 2821292 T5636 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2821292 T5636 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 2822347 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2822363 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2822366 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2822368 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2822389 T5636 oasc.ZkController.register We are http://127.0.0.1:51018/mwr/ms/collection1/ and leader is http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2822390 T5636 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51018/mwr/ms
[junit4:junit4] 2> 2822390 T5636 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2822391 T5636 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2822391 T5636 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2822395 T5636 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2822398 T5593 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2822398 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2822399 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2822843 T5593 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2822872 T5593 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51021
[junit4:junit4] 2> 2822874 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2822874 T5593 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2822875 T5593 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397
[junit4:junit4] 2> 2822875 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/'
[junit4:junit4] 2> 2822922 T5593 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/solr.xml
[junit4:junit4] 2> 2822923 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/'
[junit4:junit4] 2> 2823063 T5593 oasc.CoreContainer.<init> New CoreContainer 1484201517
[junit4:junit4] 2> 2823063 T5593 oasc.CoreContainer.load Loading cores into CoreContainer 1484201517
[junit4:junit4] 2> 2823067 T5593 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2823067 T5593 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2823068 T5593 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2823068 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2823069 T5593 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2823069 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2823070 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2823070 T5593 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2823071 T5593 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2823071 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2823091 T5593 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4:junit4] 2> 2823092 T5593 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51011/solr
[junit4:junit4] 2> 2823092 T5593 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2823100 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2823131 T5649 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a42188a name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2823132 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2823137 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2823137 T5595 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f7c7110f40007, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 2823158 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2823162 T5651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72041824 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2823162 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2823175 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2823887 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2823888 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51018_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51018/mwr/ms"}
[junit4:junit4] 2> 2823901 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2823902 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2823903 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2823912 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2824188 T5593 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51021_mwr%2Fms
[junit4:junit4] 2> 2824190 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51021_mwr%2Fms
[junit4:junit4] 2> 2824195 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2824199 T5615 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2824200 T5635 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2824201 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2824203 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2824208 T5622 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2824211 T5651 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2824211 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2824224 T5652 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2824225 T5652 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2825431 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2825432 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51021_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51021/mwr/ms"}
[junit4:junit4] 2> 2825432 T5616 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2825433 T5616 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2825472 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2825473 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2825473 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2825474 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2826229 T5652 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/collection1
[junit4:junit4] 2> 2826232 T5652 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2826236 T5652 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2826236 T5652 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2826242 T5652 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/collection1/'
[junit4:junit4] 2> 2826243 T5652 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2826244 T5652 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/collection1/lib/README' to classloader
[junit4:junit4] 2> 2826326 T5652 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2826452 T5652 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2826456 T5652 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2826467 T5652 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2827358 T5652 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2827373 T5652 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2827380 T5652 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2827409 T5652 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2827415 T5652 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2827424 T5652 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2827428 T5652 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2827429 T5652 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2827429 T5652 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2827432 T5652 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2827432 T5652 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2827432 T5652 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2827433 T5652 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372182362397/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/
[junit4:junit4] 2> 2827433 T5652 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b53df31
[junit4:junit4] 2> 2827436 T5652 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2827443 T5652 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2
[junit4:junit4] 2> 2827444 T5652 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index/
[junit4:junit4] 2> 2827445 T5652 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2827446 T5652 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index
[junit4:junit4] 2> 2827455 T5652 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2827455 T5652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2827460 T5652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2827461 T5652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2827462 T5652 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2827463 T5652 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2827464 T5652 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2827464 T5652 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2827465 T5652 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2827466 T5652 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2827466 T5652 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2827480 T5652 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2827497 T5652 oass.SolrIndexSearcher.<init> Opening Searcher@5271b2be main
[junit4:junit4] 2> 2827498 T5652 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2827499 T5652 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2827507 T5653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5271b2be main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2827511 T5652 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2827512 T5652 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51021/mwr/ms collection:collection1 shard:shard1
[junit4:junit4] 2> 2827518 T5652 oasc.ZkController.register We are http://127.0.0.1:51021/mwr/ms/collection1/ and leader is http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2827519 T5652 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51021/mwr/ms
[junit4:junit4] 2> 2827519 T5652 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2827519 T5652 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4750 name=collection1 org.apache.solr.core.SolrCore@87f866 url=http://127.0.0.1:51021/mwr/ms/collection1 node=127.0.0.1:51021_mwr%2Fms C4750_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51021_mwr%2Fms, base_url=http://127.0.0.1:51021/mwr/ms}
[junit4:junit4] 2> 2827520 T5654 C4750 P51021 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2827521 T5652 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2827522 T5654 C4750 P51021 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2827523 T5654 C4750 P51021 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2827523 T5654 C4750 P51021 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2827525 T5593 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2827525 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2827525 T5654 C4750 P51021 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2827526 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2827579 T5627 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2828016 T5593 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2828025 T5593 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51025
[junit4:junit4] 2> 2828042 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2828044 T5593 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2828045 T5593 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525
[junit4:junit4] 2> 2828045 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/'
[junit4:junit4] 2> 2828087 T5593 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/solr.xml
[junit4:junit4] 2> 2828088 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/'
[junit4:junit4] 2> 2828284 T5593 oasc.CoreContainer.<init> New CoreContainer 1849168414
[junit4:junit4] 2> 2828284 T5593 oasc.CoreContainer.load Loading cores into CoreContainer 1849168414
[junit4:junit4] 2> 2828286 T5593 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2828287 T5593 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2828287 T5593 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2828287 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2828288 T5593 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2828288 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2828288 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2828289 T5593 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2828289 T5593 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2828289 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2828307 T5593 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4:junit4] 2> 2828308 T5593 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51011/solr
[junit4:junit4] 2> 2828308 T5593 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2828310 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2828315 T5666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@206dba05 name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2828316 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2828320 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2828327 T5595 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f7c7110f40009, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 2828340 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2828345 T5668 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@714bfd28 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2828346 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2828357 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2828496 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2828497 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51021_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51021/mwr/ms"}
[junit4:junit4] 2> 2828508 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2828509 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2828513 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2828520 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2828525 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2828582 T5627 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 2828583 T5627 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:51021_mwr%252Fms&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> 2829370 T5593 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51025_mwr%2Fms
[junit4:junit4] 2> 2829372 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51025_mwr%2Fms
[junit4:junit4] 2> 2829376 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2829378 T5651 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2829378 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2829383 T5668 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2829384 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2829385 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2829391 T5622 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2829392 T5635 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2829392 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2829403 T5615 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2829403 T5669 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2829404 T5669 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2830037 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2830038 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51025_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51025/mwr/ms"}
[junit4:junit4] 2> 2830039 T5616 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2830039 T5616 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2830047 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2830047 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2830055 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2830058 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2830060 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2830429 T5669 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/collection1
[junit4:junit4] 2> 2830430 T5669 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2830431 T5669 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2830432 T5669 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2830435 T5669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/collection1/'
[junit4:junit4] 2> 2830438 T5669 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2830439 T5669 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/collection1/lib/README' to classloader
[junit4:junit4] 2> 2830546 T5669 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> ASYNC NEW_CORE C4751 name=collection1 org.apache.solr.core.SolrCore@87f866 url=http://127.0.0.1:51021/mwr/ms/collection1 node=127.0.0.1:51021_mwr%2Fms C4751_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51021_mwr%2Fms, base_url=http://127.0.0.1:51021/mwr/ms}
[junit4:junit4] 2> 2830590 T5654 C4751 P51021 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51018/mwr/ms/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2830591 T5654 C4751 P51021 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51021/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2830594 T5654 C4751 P51021 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2830609 T5654 C4751 P51021 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2830622 T5654 C4751 P51021 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2830622 T5654 C4751 P51021 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2830622 T5654 C4751 P51021 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2830622 T5654 C4751 P51021 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51018/mwr/ms/collection1/. core=collection1
[junit4:junit4] 2> 2830623 T5654 C4751 P51021 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4752 name=collection1 org.apache.solr.core.SolrCore@58e98b80 url=http://127.0.0.1:51018/mwr/ms/collection1 node=127.0.0.1:51018_mwr%2Fms C4752_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51018_mwr%2Fms, base_url=http://127.0.0.1:51018/mwr/ms, leader=true}
[junit4:junit4] 2> 2830633 T5630 C4752 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 2830675 T5669 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2830686 T5669 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> ASYNC NEW_CORE C4753 name=collection1 org.apache.solr.core.SolrCore@58e98b80 url=http://127.0.0.1:51018/mwr/ms/collection1 node=127.0.0.1:51018_mwr%2Fms C4753_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51018_mwr%2Fms, base_url=http://127.0.0.1:51018/mwr/ms, leader=true}
[junit4:junit4] 2> 2830692 T5626 C4753 P51018 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2830706 T5626 C4753 P51018 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2830709 T5626 C4753 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2830711 T5669 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2830718 T5626 C4753 P51018 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2830719 T5626 C4753 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2830722 T5626 C4753 P51018 oass.SolrIndexSearcher.<init> Opening Searcher@7014d973 realtime
[junit4:junit4] 2> 2830723 T5626 C4753 P51018 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2830724 T5626 C4753 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> ASYNC NEW_CORE C4754 name=collection1 org.apache.solr.core.SolrCore@87f866 url=http://127.0.0.1:51021/mwr/ms/collection1 node=127.0.0.1:51021_mwr%2Fms C4754_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51021_mwr%2Fms, base_url=http://127.0.0.1:51021/mwr/ms}
[junit4:junit4] 2> 2830743 T5654 C4754 P51021 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2830743 T5654 C4754 P51021 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2830757 T5629 C4753 P51018 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2830758 T5629 C4753 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=11
[junit4:junit4] 2> 2830759 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2830759 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2830760 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2830774 T5627 C4753 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 2830788 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2830797 T5654 C4754 P51021 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769
[junit4:junit4] 2> 2830799 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769 lockFactory=org.apache.lucene.store.NativeFSLockFactory@351a343b; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2830813 T5630 C4753 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2830828 T5654 C4754 P51021 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2830831 T5654 C4754 P51021 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2830832 T5654 C4754 P51021 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2830850 T5654 C4754 P51021 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2830853 T5654 C4754 P51021 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2830854 T5654 C4754 P51021 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2830855 T5654 C4754 P51021 oass.SolrIndexSearcher.<init> Opening Searcher@7a3a3ede main
[junit4:junit4] 2> 2830857 T5653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a3a3ede main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2830857 T5654 C4754 P51021 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769;done=true>>]
[junit4:junit4] 2> 2830858 T5654 C4754 P51021 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769
[junit4:junit4] 2> 2830858 T5654 C4754 P51021 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index.20130626054610769
[junit4:junit4] 2> 2830860 T5654 C4754 P51021 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2830860 T5654 C4754 P51021 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2830860 T5654 C4754 P51021 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2830860 T5654 C4754 P51021 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2830865 T5654 C4754 P51021 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2831568 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2831570 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51021_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51021/mwr/ms"}
[junit4:junit4] 2> 2831578 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2831578 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2831578 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2831579 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2831579 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2831689 T5669 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2831709 T5669 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2831713 T5669 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2831748 T5669 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2831768 T5669 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2831776 T5669 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2831785 T5669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2831786 T5669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2831786 T5669 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2831789 T5669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2831790 T5669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2831790 T5669 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2831791 T5669 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372182367525/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/
[junit4:junit4] 2> 2831791 T5669 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b53df31
[junit4:junit4] 2> 2831792 T5669 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2831793 T5669 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3
[junit4:junit4] 2> 2831793 T5669 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index/
[junit4:junit4] 2> 2831794 T5669 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2831795 T5669 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index
[junit4:junit4] 2> 2831803 T5669 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2831804 T5669 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2831809 T5669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2831810 T5669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2831811 T5669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2831812 T5669 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2831813 T5669 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2831813 T5669 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2831816 T5669 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2831817 T5669 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2831818 T5669 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2831835 T5669 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2831853 T5669 oass.SolrIndexSearcher.<init> Opening Searcher@6619bf6a main
[junit4:junit4] 2> 2831856 T5669 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2831856 T5669 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2831867 T5672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6619bf6a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2831911 T5669 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2831915 T5669 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51025/mwr/ms collection:collection1 shard:shard1
[junit4:junit4] 2> 2831932 T5669 oasc.ZkController.register We are http://127.0.0.1:51025/mwr/ms/collection1/ and leader is http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2831932 T5669 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51025/mwr/ms
[junit4:junit4] 2> 2831932 T5669 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2831933 T5669 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4755 name=collection1 org.apache.solr.core.SolrCore@730172d5 url=http://127.0.0.1:51025/mwr/ms/collection1 node=127.0.0.1:51025_mwr%2Fms C4755_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51025_mwr%2Fms, base_url=http://127.0.0.1:51025/mwr/ms}
[junit4:junit4] 2> 2831934 T5673 C4755 P51025 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2831935 T5673 C4755 P51025 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2831935 T5673 C4755 P51025 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2831935 T5673 C4755 P51025 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2831938 T5673 C4755 P51025 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2831962 T5669 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2831962 T5628 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2831977 T5593 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2831978 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2831978 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2832433 T5593 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2832446 T5593 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51032
[junit4:junit4] 2> 2832448 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2832448 T5593 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2832449 T5593 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985
[junit4:junit4] 2> 2832450 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/'
[junit4:junit4] 2> 2832500 T5593 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/solr.xml
[junit4:junit4] 2> 2832503 T5593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/'
[junit4:junit4] 2> 2832653 T5593 oasc.CoreContainer.<init> New CoreContainer 73051651
[junit4:junit4] 2> 2832654 T5593 oasc.CoreContainer.load Loading cores into CoreContainer 73051651
[junit4:junit4] 2> 2832656 T5593 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2832656 T5593 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2832657 T5593 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2832657 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2832658 T5593 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2832658 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2832659 T5593 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2832659 T5593 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2832660 T5593 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2832660 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2832679 T5593 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4:junit4] 2> 2832680 T5593 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51011/solr
[junit4:junit4] 2> 2832681 T5593 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2832683 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2832689 T5685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2536c693 name:ZooKeeperConnection Watcher:127.0.0.1:51011 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2832690 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2832696 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2832720 T5593 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2832724 T5687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6054b913 name:ZooKeeperConnection Watcher:127.0.0.1:51011/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2832725 T5593 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2832736 T5593 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2833093 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2833094 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51025_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51025/mwr/ms"}
[junit4:junit4] 2> 2833113 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833114 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833114 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833115 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833114 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833114 T5687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833747 T5593 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51032_mwr%2Fms
[junit4:junit4] 2> 2833750 T5593 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51032_mwr%2Fms
[junit4:junit4] 2> 2833757 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833758 T5687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833757 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833757 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2833762 T5651 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833763 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2833765 T5635 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833765 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2833770 T5615 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833776 T5687 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833790 T5622 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833792 T5668 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2833792 T5688 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2833793 T5688 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2833967 T5628 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2833968 T5628 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:51025_mwr%252Fms&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2006
[junit4:junit4] 2> 2834640 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2834644 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51032_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51032/mwr/ms"}
[junit4:junit4] 2> 2834645 T5616 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2834646 T5616 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2834654 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834655 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834655 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834659 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834663 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834663 T5687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2834797 T5688 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/collection1
[junit4:junit4] 2> 2834798 T5688 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2834800 T5688 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2834801 T5688 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2834803 T5688 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/collection1/'
[junit4:junit4] 2> 2834805 T5688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2834812 T5688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/collection1/lib/README' to classloader
[junit4:junit4] 2> 2834894 T5688 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2834980 T5688 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2834984 T5688 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2834996 T5688 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2835909 T5688 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2835921 T5688 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2835927 T5688 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2835968 T5688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2835976 T5688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> ASYNC NEW_CORE C4756 name=collection1 org.apache.solr.core.SolrCore@730172d5 url=http://127.0.0.1:51025/mwr/ms/collection1 node=127.0.0.1:51025_mwr%2Fms C4756_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51025_mwr%2Fms, base_url=http://127.0.0.1:51025/mwr/ms}
[junit4:junit4] 2> 2835976 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51018/mwr/ms/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2835977 T5673 C4756 P51025 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51025/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2835978 T5673 C4756 P51025 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2835978 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2835978 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2835979 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2835979 T5673 C4756 P51025 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2835979 T5673 C4756 P51025 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51018/mwr/ms/collection1/. core=collection1
[junit4:junit4] 2> 2835980 T5673 C4756 P51025 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4757 name=collection1 org.apache.solr.core.SolrCore@58e98b80 url=http://127.0.0.1:51018/mwr/ms/collection1 node=127.0.0.1:51018_mwr%2Fms C4757_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51018_mwr%2Fms, base_url=http://127.0.0.1:51018/mwr/ms, leader=true}
[junit4:junit4] 2> 2835991 T5629 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2835999 T5627 C4757 P51018 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2836014 T5627 C4757 P51018 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2836015 T5627 C4757 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2836017 T5627 C4757 P51018 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2836017 T5627 C4757 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 2836019 T5673 C4756 P51025 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2836019 T5673 C4756 P51025 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2836022 T5631 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 2836023 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 2836023 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2836024 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2836027 T5628 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 2836035 T5688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2836036 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2836038 T5673 C4756 P51025 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016
[junit4:junit4] 2> 2836039 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2807cb38; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2836043 T5688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2836045 T5688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2836045 T5688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2836047 T5688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2836047 T5688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2836048 T5688 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2836048 T5688 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372182371985/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/
[junit4:junit4] 2> 2836048 T5688 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b53df31
[junit4:junit4] 2> 2836049 T5688 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2836050 T5688 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4
[junit4:junit4] 2> 2836051 T5688 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index/
[junit4:junit4] 2> 2836053 T5688 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2836055 T5688 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index
[junit4:junit4] 2> 2836063 T5688 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2836064 T5688 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2836064 T5626 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 2836076 T5673 C4756 P51025 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2836076 T5688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2836077 T5688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2836078 T5688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2836079 T5688 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2836080 T5673 C4756 P51025 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2836080 T5673 C4756 P51025 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2836080 T5688 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2836081 T5688 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2836086 T5688 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2836087 T5673 C4756 P51025 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2836087 T5688 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2836089 T5673 C4756 P51025 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2836090 T5688 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2836091 T5673 C4756 P51025 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2836092 T5673 C4756 P51025 oass.SolrIndexSearcher.<init> Opening Searcher@3dfd4bf6 main
[junit4:junit4] 2> 2836095 T5672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3dfd4bf6 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 2836095 T5673 C4756 P51025 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016;done=true>>]
[junit4:junit4] 2> 2836096 T5673 C4756 P51025 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016
[junit4:junit4] 2> 2836096 T5673 C4756 P51025 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index.20130626054616016
[junit4:junit4] 2> 2836097 T5673 C4756 P51025 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2836098 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2836098 T5673 C4756 P51025 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2836098 T5673 C4756 P51025 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2836101 T5673 C4756 P51025 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2836113 T5688 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2836135 T5688 oass.SolrIndexSearcher.<init> Opening Searcher@22db002d main
[junit4:junit4] 2> 2836141 T5688 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2836142 T5688 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2836148 T5691 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22db002d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2836152 T5688 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2836152 T5688 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51032/mwr/ms collection:collection1 shard:shard1
[junit4:junit4] 2> 2836162 T5688 oasc.ZkController.register We are http://127.0.0.1:51032/mwr/ms/collection1/ and leader is http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2836163 T5688 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51032/mwr/ms
[junit4:junit4] 2> 2836163 T5688 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2836163 T5688 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4758 name=collection1 org.apache.solr.core.SolrCore@5974f895 url=http://127.0.0.1:51032/mwr/ms/collection1 node=127.0.0.1:51032_mwr%2Fms C4758_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51032_mwr%2Fms, base_url=http://127.0.0.1:51032/mwr/ms}
[junit4:junit4] 2> 2836164 T5692 C4758 P51032 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2836166 T5688 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2836169 T5593 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2836170 T5593 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2836170 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2836185 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2836185 T5692 C4758 P51032 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2836185 T5692 C4758 P51032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2836195 T5692 C4758 P51032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2836199 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2836209 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2836211 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51025_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51025/mwr/ms"}
[junit4:junit4] 2> 2836216 T5593 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 2836218 T5593 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 2836219 T5629 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2836220 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2836221 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51032_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51032/mwr/ms"}
[junit4:junit4] 2> 2836231 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2836232 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2836238 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2836239 T5687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2836239 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2836242 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2837221 T5629 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 2837222 T5629 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:51032_mwr%252Fms&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 2837224 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2838228 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C4758_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51032_mwr%2Fms, base_url=http://127.0.0.1:51032/mwr/ms}
[junit4:junit4] 2> 2839228 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51018/mwr/ms/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2839228 T5692 C4758 P51032 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51032/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2839230 T5692 C4758 P51032 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2839230 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2839230 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2839231 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2839231 T5692 C4758 P51032 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2839231 T5692 C4758 P51032 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51018/mwr/ms/collection1/. core=collection1
[junit4:junit4] 2> 2839231 T5692 C4758 P51032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2839251 T5629 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 2839253 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2839256 T5631 C4757 P51018 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2839261 T5631 C4757 P51018 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2839262 T5631 C4757 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2839263 T5631 C4757 P51018 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2839263 T5631 C4757 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 2839265 T5692 C4758 P51032 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2839265 T5692 C4758 P51032 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2839268 T5626 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 2839270 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 2839270 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2839270 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2839274 T5627 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 2839274 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2839276 T5692 C4758 P51032 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255
[junit4:junit4] 2> 2839277 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b52f65f; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2839281 T5630 C4757 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 2839284 T5692 C4758 P51032 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2839287 T5692 C4758 P51032 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2839287 T5692 C4758 P51032 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2839292 T5692 C4758 P51032 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2839294 T5692 C4758 P51032 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2839294 T5692 C4758 P51032 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2839295 T5692 C4758 P51032 oass.SolrIndexSearcher.<init> Opening Searcher@20fcb50f main
[junit4:junit4] 2> 2839297 T5691 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20fcb50f main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 2839298 T5692 C4758 P51032 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255;done=true>>]
[junit4:junit4] 2> 2839298 T5692 C4758 P51032 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255
[junit4:junit4] 2> 2839298 T5692 C4758 P51032 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index.20130626054619255
[junit4:junit4] 2> 2839304 T5692 C4758 P51032 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2839305 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2839305 T5692 C4758 P51032 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2839305 T5692 C4758 P51032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2839307 T5692 C4758 P51032 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2840259 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2840764 T5616 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2840766 T5616 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51032_mwr%2Fms",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51032/mwr/ms"}
[junit4:junit4] 2> 2840779 T5622 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2840779 T5668 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2840780 T5687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2840779 T5635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2840780 T5651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2840780 T5615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2841263 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2841264 T5593 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C4759 name=collection1 org.apache.solr.core.SolrCore@678750b2 url=http://127.0.0.1:51014/mwr/ms/collection1 node=127.0.0.1:51014_mwr%2Fms C4759_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51014_mwr%2Fms, base_url=http://127.0.0.1:51014/mwr/ms, leader=true}
[junit4:junit4] 2> 2841282 T5608 C4759 P51014 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2841290 T5608 C4759 P51014 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2841292 T5608 C4759 P51014 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2841298 T5608 C4759 P51014 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2841298 T5608 C4759 P51014 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2841300 T5608 C4759 P51014 oass.SolrIndexSearcher.<init> Opening Searcher@7447e6ba main
[junit4:junit4] 2> 2841300 T5608 C4759 P51014 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2841302 T5619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7447e6ba main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2841302 T5608 C4759 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> C4756_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51025_mwr%2Fms, base_url=http://127.0.0.1:51025/mwr/ms}
[junit4:junit4] 2> 2841340 T5661 C4756 P51025 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51018/mwr/ms/collection1/, StdNode: http://127.0.0.1:51021/mwr/ms/collection1/, StdNode: http://127.0.0.1:51025/mwr/ms/collection1/, StdNode: http://127.0.0.1:51032/mwr/ms/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 2841381 T5663 C4756 P51025 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2841386 T5630 C4757 P51018 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2841392 T5630 C4757 P51018 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2841393 T5630 C4757 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2841418 T5663 C4756 P51025 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2841419 T5663 C4756 P51025 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2841427 T5630 C4757 P51018 oass.SolrIndexSearcher.<init> Opening Searcher@73f38d18 main
[junit4:junit4] 2> ASYNC NEW_CORE C4760 name=collection1 org.apache.solr.core.SolrCore@87f866 url=http://127.0.0.1:51021/mwr/ms/collection1 node=127.0.0.1:51021_mwr%2Fms C4760_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51021_mwr%2Fms, base_url=http://127.0.0.1:51021/mwr/ms}
[junit4:junit4] 2> 2841429 T5647 C4760 P51021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2841430 T5637 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73f38d18 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2841430 T5630 C4757 P51018 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2841431 T5630 C4757 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 45
[junit4:junit4] 2> ASYNC NEW_CORE C4761 name=collection1 org.apache.solr.core.SolrCore@730172d5 url=http://127.0.0.1:51025/mwr/ms/collection1 node=127.0.0.1:51025_mwr%2Fms C4761_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51025_mwr%2Fms, base_url=http://127.0.0.1:51025/mwr/ms}
[junit4:junit4] 2> 2841456 T5663 C4761 P51025 oass.SolrIndexSearcher.<init> Opening Searcher@2003c5f2 main
[junit4:junit4] 2> 2841457 T5663 C4761 P51025 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C4762 name=collection1 org.apache.solr.core.SolrCore@87f866 url=http://127.0.0.1:51021/mwr/ms/collection1 node=127.0.0.1:51021_mwr%2Fms C4762_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51021_mwr%2Fms, base_url=http://127.0.0.1:51021/mwr/ms}
[junit4:junit4] 2> 2841457 T5647 C4762 P51021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2841458 T5647 C4762 P51021 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2841458 T5672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2003c5f2 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 2841460 T5663 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 80
[junit4:junit4] 2> ASYNC NEW_CORE C4763 name=collection1 org.apache.solr.core.SolrCore@5974f895 url=http://127.0.0.1:51032/mwr/ms/collection1 node=127.0.0.1:51032_mwr%2Fms C4763_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51032_mwr%2Fms, base_url=http://127.0.0.1:51032/mwr/ms}
[junit4:junit4] 2> 2841461 T5681 C4763 P51032 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2841463 T5647 C4762 P51021 oass.SolrIndexSearcher.<init> Opening Searcher@30caf364 main
[junit4:junit4] 2> 2841466 T5647 C4762 P51021 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2841467 T5653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@30caf364 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2841468 T5647 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4] 2> 2841477 T5681 C4763 P51032 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2841477 T5681 C4763 P51032 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2841480 T5681 C4763 P51032 oass.SolrIndexSearcher.<init> Opening Searcher@58c99850 main
[junit4:junit4] 2> 2841481 T5681 C4763 P51032 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2841482 T5691 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58c99850 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 2841483 T5681 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> 2841484 T5661 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 144
[junit4:junit4] 2> 2841487 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C4764 name=collection1 org.apache.solr.core.SolrCore@58e98b80 url=http://127.0.0.1:51018/mwr/ms/collection1 node=127.0.0.1:51018_mwr%2Fms C4764_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51018_mwr%2Fms, base_url=http://127.0.0.1:51018/mwr/ms, leader=true}
[junit4:junit4] 2> 2841492 T5629 C4764 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 2841497 T5640 C4762 P51021 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 2841504 T5664 C4761 P51025 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 2841510 T5682 C4763 P51032 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C4765 name=collection1 org.apache.solr.core.SolrCore@678750b2 url=http://127.0.0.1:51014/mwr/ms/collection1 node=127.0.0.1:51014_mwr%2Fms C4765_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51014_mwr%2Fms, base_url=http://127.0.0.1:51014/mwr/ms, leader=true}
[junit4:junit4] 2> 2843518 T5608 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438837514956701696)} 0 3
[junit4:junit4] 2> 2843553 T5664 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={update.distrib=FROMLEADER&_version_=-1438837514969284608&update.from=http://127.0.0.1:51018/mwr/ms/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438837514969284608)} 0 3
[junit4:junit4] 2> 2843553 T5683 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={update.distrib=FROMLEADER&_version_=-1438837514969284608&update.from=http://127.0.0.1:51018/mwr/ms/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438837514969284608)} 0 2
[junit4:junit4] 2> 2843556 T5640 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={update.distrib=FROMLEADER&_version_=-1438837514969284608&update.from=http://127.0.0.1:51018/mwr/ms/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438837514969284608)} 0 2
[junit4:junit4] 2> 2843557 T5629 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438837514969284608)} 0 29
[junit4:junit4] 2> 2843558 T5682 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 36
[junit4:junit4] 2> 2843567 T5609 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[0 (1438837515008081920)]} 0 3
[junit4:junit4] 2> 2843584 T5659 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438837515020664832)]} 0 2
[junit4:junit4] 2> 2843589 T5645 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438837515020664832)]} 0 2
[junit4:junit4] 2> 2843600 T5678 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438837515020664832)]} 0 9
[junit4:junit4] 2> 2843602 T5631 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51021/mwr/ms/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438837515020664832)]} 0 25
[junit4:junit4] 2> 2843603 T5644 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[0]} 0 31
[junit4:junit4] 2> 2843608 T5610 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[1 (1438837515052122112)]} 0 2
[junit4:junit4] 2> 2843615 T5646 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438837515056316416)]} 0 0
[junit4:junit4] 2> 2843616 T5660 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438837515056316416)]} 0 1
[junit4:junit4] 2> 2843617 T5679 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438837515056316416)]} 0 1
[junit4:junit4] 2> 2843618 T5628 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[1 (1438837515056316416)]} 0 7
[junit4:junit4] 2> 2843625 T5611 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[2 (1438837515069947904)]} 0 1
[junit4:junit4] 2> 2843631 T5626 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:51021/mwr/ms/collection1/]
[junit4:junit4] 2> 2843632 T5626 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51021/mwr/ms/collection1/ against:[http://127.0.0.1:51021/mwr/ms/collection1/] result:true
[junit4:junit4] 2> 2843632 T5626 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51025/mwr/ms/collection1/ against:[http://127.0.0.1:51021/mwr/ms/collection1/] result:false
[junit4:junit4] 2> 2843632 T5626 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51032/mwr/ms/collection1/ against:[http://127.0.0.1:51021/mwr/ms/collection1/] result:false
[junit4:junit4] 2> 2843640 T5662 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&wt=javabin&version=2} {add=[2 (1438837515079385088)]} 0 2
[junit4:junit4] 2> 2843646 T5680 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&wt=javabin&version=2} {add=[2 (1438837515079385088)]} 0 3
[junit4:junit4] 2> 2843647 T5626 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51021/mwr/ms/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&wt=javabin&version=2} {add=[2 (1438837515079385088)]} 0 16
[junit4:junit4] 2> 2843648 T5647 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&wt=javabin&version=2} {add=[2]} 0 20
[junit4:junit4] 2> 2843652 T5606 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={wt=javabin&version=2} {add=[3 (1438837515098259456)]} 0 1
[junit4:junit4] 2> 2843655 T5627 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:51025/mwr/ms/collection1/, http://127.0.0.1:51021/mwr/ms/collection1/]
[junit4:junit4] 2> 2843655 T5627 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51021/mwr/ms/collection1/ against:[http://127.0.0.1:51025/mwr/ms/collection1/, http://127.0.0.1:51021/mwr/ms/collection1/] result:true
[junit4:junit4] 2> 2843656 T5627 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51025/mwr/ms/collection1/ against:[http://127.0.0.1:51025/mwr/ms/collection1/, http://127.0.0.1:51021/mwr/ms/collection1/] result:true
[junit4:junit4] 2> 2843656 T5627 C4764 P51018 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51032/mwr/ms/collection1/ against:[http://127.0.0.1:51025/mwr/ms/collection1/, http://127.0.0.1:51021/mwr/ms/collection1/] result:false
[junit4:junit4] 2> 2843661 T5681 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={distrib.from=http://127.0.0.1:51018/mwr/ms/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&wt=javabin&version=2} {add=[3 (1438837515104550912)]} 0 1
[junit4:junit4] 2> 2843662 T5627 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={test.distrib.skip.servers=http://127.0.0.1:51021/mwr/ms/collection1/&test.distrib.skip.servers=http://127.0.0.1:51025/mwr/ms/collection1/&wt=javabin&version=2} {add=[3 (1438837515104550912)]} 0 7
[junit4:junit4] 2> 2843665 T5607 C4765 P51014 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2843708 T5607 C4765 P51014 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8467eb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2843715 T5607 C4765 P51014 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2843720 T5607 C4765 P51014 oass.SolrIndexSearcher.<init> Opening Searcher@71423832 main
[junit4:junit4] 2> 2843720 T5607 C4765 P51014 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2843722 T5619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71423832 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 2843723 T5607 C4765 P51014 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 2843726 T5683 C4763 P51032 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51018/mwr/ms/collection1/, StdNode: http://127.0.0.1:51021/mwr/ms/collection1/, StdNode: http://127.0.0.1:51025/mwr/ms/collection1/, StdNode: http://127.0.0.1:51032/mwr/ms/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 2843737 T5630 C4764 P51018 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2843739 T5663 C4761 P51025 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2843749 T5682 C4763 P51032 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2843760 T5640 C4762 P51021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2843842 T5682 C4763 P51032 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2373f61b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 2843843 T5682 C4763 P51032 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 2843852 T5682 C4763 P51032 oass.SolrIndexSearcher.<init> Opening Searcher@3accab5b main
[junit4:junit4] 2> 2843852 T5682 C4763 P51032 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2843856 T5630 C4764 P51018 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ca1d86d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 2843857 T5630 C4764 P51018 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 2843862 T5630 C4764 P51018 oass.SolrIndexSearcher.<init> Opening Searcher@2aa280e7 main
[junit4:junit4] 2> 2843866 T5691 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3accab5b main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 2843870 T5637 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2aa280e7 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 2843870 T5630 C4764 P51018 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2843871 T5630 C4764 P51018 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 134
[junit4:junit4] 2> 2843872 T5682 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 123
[junit4:junit4] 2> 2843879 T5640 C4762 P51021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@275f19ea; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2843881 T5640 C4762 P51021 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2843871 T5663 C4761 P51025 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372182352107/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39c4a11d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2843882 T5663 C4761 P51025 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2843886 T5640 C4762 P51021 oass.SolrIndexSearcher.<init> Opening Searcher@d215ce0 main
[junit4:junit4] 2> 2843887 T5663 C4761 P51025 oass.SolrIndexSearcher.<init> Opening Searcher@79ba16f5 main
[junit4:junit4] 2> 2843888 T5653 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d215ce0 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):c2)}
[junit4:junit4] 2> 2843889 T5640 C4762 P51021 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2843890 T5640 C4762 P51021 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 130
[junit4:junit4] 2> 2843890 T5672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79ba16f5 main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):c3)}
[junit4:junit4] 2> 2843891 T5663 C4761 P51025 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2843892 T5663 C4761 P51025 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 153
[junit4:junit4] 2> 2843893 T5683 C4763 P51032 oasup.LogUpdateProcessor.finish [collection1] webapp=/mwr/ms path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 167
[junit4:junit4] 2> 2843894 T5593 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 2843895 T5593 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2843897 T5593 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 2843900 T5629 C4764 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 2843903 T5643 C4762 P51021 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 2843906 T5661 C4761 P51025 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=3 status=0 QTime=0
[junit4:junit4] 2> 2843910 T5678 C4763 P51032 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 2843911 T5593 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2843937 T5659 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 2843937 T5659 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2843953 T5628 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 2843953 T5628 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2843954 T5628 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms START replicas=[http://127.0.0.1:51021/mwr/ms/collection1/, http://127.0.0.1:51025/mwr/ms/collection1/, http://127.0.0.1:51032/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2843960 T5679 C4763 P51032 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2843960 T5660 C4761 P51025 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2843962 T5644 C4762 P51021 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2843962 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Received 5 versions from 127.0.0.1:51032/mwr/ms/collection1/
[junit4:junit4] 2> 2843963 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Our versions are newer. ourLowThreshold=1438837514969284608 otherHigh=1438837515079385088
[junit4:junit4] 2> 2843963 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Received 4 versions from 127.0.0.1:51025/mwr/ms/collection1/
[junit4:junit4] 2> 2843964 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Our versions are newer. ourLowThreshold=1438837514969284608 otherHigh=1438837515079385088
[junit4:junit4] 2> 2843964 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Received 3 versions from 127.0.0.1:51021/mwr/ms/collection1/
[junit4:junit4] 2> 2843964 T5628 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms Our versions are newer. ourLowThreshold=1438837514969284608 otherHigh=1438837515056316416
[junit4:junit4] 2> 2843965 T5628 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51018/mwr/ms DONE. sync succeeded
[junit4:junit4] 2> 2843965 T5628 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2843966 T5628 oasc.SyncStrategy.syncToMe http://127.0.0.1:51018/mwr/ms/collection1/: try and ask http://127.0.0.1:51021/mwr/ms/collection1/ to sync
[junit4:junit4] 2> 2843975 T5628 oasc.SyncStrategy.syncToMe http://127.0.0.1:51018/mwr/ms/collection1/: try and ask http://127.0.0.1:51025/mwr/ms/collection1/ to sync
[junit4:junit4] 2> 2843975 T5628 oasc.SyncStrategy.syncToMe http://127.0.0.1:51018/mwr/ms/collection1/: try and ask http://127.0.0.1:51032/mwr/ms/collection1/ to sync
[junit4:junit4] 2> 2843978 T5646 C4762 P51021 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51021/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2843981 T5681 C4763 P51032 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51032/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2843983 T5662 C4761 P51025 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51025/mwr/ms START replicas=[http://127.0.0.1:51018/mwr/ms/collection1/] nUpdates=100
[junit4:junit4] 2> 2843984 T5626 C4764 P51018 oasc.SolrCore.execute [collection1] webapp=/mwr/ms path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2843991 T5646 C4762 P51021 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51021/mwr/ms Received 5 versions from 127.0.0.1:51018/mwr/ms/collection1/
[junit4:junit4] 2> 2843991 T5646 C4762 P51021 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:51021/mwr/ms Requesting updates from 127.0.0.1:51018/mwr/ms/collection1/n=2 versions=[1438837515104550912, 1438837515079385088]
[junit4:junit4] 2> 2843993 T5629 C4764 P51018 oasc.SolrCore.execute [collection1]
[...truncated too long message...]
4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89927342628274186-4-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89927342628274190-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89927342628274188-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51032_mwr%2Fms",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51032/mwr/ms"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89927342628274179-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51014_mwr%2Fms",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51014/mwr/ms"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89927342628274188-127.0.0.1:51032_mwr%2Fms-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89927342628274186-127.0.0.1:51025_mwr%2Fms-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89927342628274190-127.0.0.1:51018_mwr%2Fms-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89927342628274179-127.0.0.1:51014_mwr%2Fms-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89927342628274179-127.0.0.1:51014_mwr%2Fms-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=6E9B481AEB21F1C2 -Dtests.slow=true -Dtests.locale=ar_LB -Dtests.timezone=NZ -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 119s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51018/mwr/ms/collection1lastClient and got 265 from http://127.0.0.1:51025/mwr/ms/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([6E9B481AEB21F1C2:EF7DC6029C7E91FE]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 2931348 T5593 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 119238 T5592 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=MockRandom), text=MockVariableIntBlock(baseBlockSize=75), _version_=PostingsFormat(name=MockRandom), rnd_b=PostingsFormat(name=TestBloomFilteredLucene41Postings), intDefault=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=MockRandom), id=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_t=PostingsFormat(name=MockRandom), range_facet_sl=MockVariableIntBlock(baseBlockSize=75), range_facet_si=PostingsFormat(name=Memory doPackFST= false), other_tl1=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=Memory doPackFST= false), a_si=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_LB, timezone=NZ
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=1,free=80741744,total=220418048
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestLMDirichletSimilarityFactory, ClusterStateTest, PluginInfoTest, AutoCommitTest, TestSchemaSimilarityResource, TestMaxScoreQueryParser, TermVectorComponentTest, TestCSVLoader, TestCoreDiscovery, EchoParamsTest, TestLuceneMatchVersion, TestUtils, TestFastLRUCache, TestExtendedDismaxParser, TestRTGBase, SolrCoreCheckLockOnStartupTest, ClusterStateUpdateTest, TestCloudManagedSchema, StandardRequestHandlerTest, SolrCoreTest, ShardSplitTest, TestFaceting, TestRandomDVFaceting, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, ZkSolrClientTest, TestPhraseSuggestions, TestSerializedLuceneMatchVersion, BasicDistributedZkTest, TestQuerySenderListener, ResponseLogComponentTest, TestNumberUtils, DirectUpdateHandlerOptimizeTest, FastVectorHighlighterTest, SuggesterWFSTTest, ZkNodePropsTest, TestElisionMultitermQuery, TestSolrCoreProperties, TestSuggestSpellingConverter, CopyFieldTest, TermVectorComponentDistributedTest, CoreAdminHandlerTest, DistanceFunctionTest, TestSystemIdResolver, DistributedQueryElevationComponentTest, ShowFileRequestHandlerTest, TestStressRecovery, TestGroupingSearch, CurrencyFieldOpenExchangeTest, QueryEqualityTest, TestFieldCollectionResource, TestConfig, NotRequiredUniqueKeyTest, TestSweetSpotSimilarityFactory, TestRealTimeGet, HardAutoCommitTest, DOMUtilTest, QueryElevationComponentTest, MinimalSchemaTest, SimplePostToolTest, TestReplicationHandler, TestLFUCache, TestDocSet, TestPostingsSolrHighlighter, TestSchemaVersionResource, TestSolrXMLSerializer, TestZkChroot, RAMDirectoryFactoryTest, TestCharFilters, AliasIntegrationTest, UpdateRequestProcessorFactoryTest, LeaderElectionTest, PreAnalyzedUpdateProcessorTest, DisMaxRequestHandlerTest, TestTrie, TestRangeQuery, TestManagedSchemaFieldResource, PingRequestHandlerTest, ShardRoutingCustomTest, TestJoin, TestValueSourceCache, TestReversedWildcardFilterFactory, TestDistributedSearch, TestLMJelinekMercerSimilarityFactory, DocumentAnalysisRequestHandlerTest, TestDynamicFieldCollectionResource, TestJmxMonitoredMap, TestLazyCores, SolrPluginUtilsTest, PreAnalyzedFieldTest, PathHierarchyTokenizerFactoryTest, SpellCheckCollatorTest, FileBasedSpellCheckerTest, HighlighterTest, ExternalFileFieldSortTest, DebugComponentTest, TestSchemaResource, StatsComponentTest, TestRecovery, DirectSolrConnectionTest, CacheHeaderTest, TestLRUCache, TestSolrDeletionPolicy1, BadIndexSchemaTest, TestAtomicUpdateErrorCases, XmlUpdateRequestHandlerTest, TestReload, SchemaVersionSpecificBehaviorTest, ZkControllerTest, TestPseudoReturnFields, SuggesterFSTTest, ChaosMonkeyNothingIsSafeTest, TestAddFieldRealTimeGet, SolrTestCaseJ4Test, TestPropInjectDefaults, TestBinaryResponseWriter, OverseerTest, ZkCLITest, ShardRoutingTest, ConvertedLegacyTest, TestIBSimilarityFactory, DateMathParserTest, RegexBoostProcessorTest, ResourceLoaderTest, TimeZoneUtilsTest, TestIndexSearcher, ReturnFieldsTest, TestFuzzyAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, SolrIndexSplitterTest, TermsComponentTest, TestDynamicFieldResource, DirectUpdateHandlerTest, TestQuerySenderNoQuery, TestCloudManagedSchemaAddField, FieldAnalysisRequestHandlerTest, DefaultValueUpdateProcessorTest, CollectionsAPIDistributedZkTest, TestManagedSchema, SOLR749Test, UUIDFieldTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed in 120.05s, 1 test, 1 failure <<< FAILURES!
[...truncated 453 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 107 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 596 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/596/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<130> but was:<26>
Stack Trace:
java.lang.AssertionError: expected:<130> but was:<26>
at __randomizedtesting.SeedInfo.seed([10E034D761A8A91B:9106BACF16F7C927]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9372 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 712125 T1769 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_ro/m
[junit4:junit4] 2> 712131 T1769 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1372203311977
[junit4:junit4] 2> 712134 T1769 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 712139 T1770 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 712240 T1769 oasc.ZkTestServer.run start zk server on port:49642
[junit4:junit4] 2> 712242 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 712253 T1776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1afdbb08 name:ZooKeeperConnection Watcher:127.0.0.1:49642 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 712253 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 712253 T1769 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 712263 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 712268 T1778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b41ad23 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 712268 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 712268 T1769 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 712277 T1769 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 712290 T1769 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 712297 T1769 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 712304 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 712304 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 712315 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 712315 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 712323 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 712324 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 712331 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 712332 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 712339 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 712340 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 712347 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 712347 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 712360 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 712360 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 712368 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 712368 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 712376 T1769 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 712376 T1769 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 712823 T1769 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 712831 T1769 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49645
[junit4:junit4] 2> 712831 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 712832 T1769 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 712832 T1769 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231
[junit4:junit4] 2> 712833 T1769 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/solr.xml
[junit4:junit4] 2> 712833 T1769 oasc.CoreContainer.<init> New CoreContainer 382108976
[junit4:junit4] 2> 712834 T1769 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/'
[junit4:junit4] 2> 712834 T1769 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/'
[junit4:junit4] 2> 712985 T1769 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 712986 T1769 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 712986 T1769 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 712987 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 712987 T1769 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 712988 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 712988 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 712989 T1769 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 712989 T1769 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 712990 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 713014 T1769 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 713015 T1769 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49642/solr
[junit4:junit4] 2> 713016 T1769 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 713017 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 713021 T1789 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@756c05c name:ZooKeeperConnection Watcher:127.0.0.1:49642 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 713022 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 713027 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 713047 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 713052 T1791 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48357908 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 713052 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 713056 T1769 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 713065 T1769 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 713073 T1769 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 713077 T1769 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49645_e_ro%2Fm
[junit4:junit4] 2> 713080 T1769 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49645_e_ro%2Fm
[junit4:junit4] 2> 713093 T1769 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 713107 T1769 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 713114 T1769 oasc.Overseer.start Overseer (id=89928716254773251-127.0.0.1:49645_e_ro%2Fm-n_0000000000) starting
[junit4:junit4] 2> 713121 T1769 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 713133 T1792 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 713137 T1793 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 713147 T1769 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 713153 T1769 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 713159 T1769 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 713171 T1794 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 713172 T1794 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 714649 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 714651 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49645_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49645/e_ro/m"}
[junit4:junit4] 2> 714651 T1792 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 714652 T1792 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 714667 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 715176 T1794 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/collection1
[junit4:junit4] 2> 715176 T1794 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 715178 T1794 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 715179 T1794 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 715182 T1794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/collection1/'
[junit4:junit4] 2> 715184 T1794 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 715185 T1794 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/collection1/lib/README' to classloader
[junit4:junit4] 2> 715265 T1794 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 715363 T1794 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 715366 T1794 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 715372 T1794 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 716201 T1794 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 716212 T1794 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 716217 T1794 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 716248 T1794 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 716257 T1794 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 716269 T1794 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 716272 T1794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 716272 T1794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 716272 T1794 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 716275 T1794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 716275 T1794 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 716276 T1794 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 716276 T1794 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1372203312231/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data/
[junit4:junit4] 2> 716276 T1794 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ec40b92
[junit4:junit4] 2> 716277 T1794 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 716278 T1794 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data
[junit4:junit4] 2> 716278 T1794 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data/index/
[junit4:junit4] 2> 716279 T1794 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 716280 T1794 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data/index
[junit4:junit4] 2> 716285 T1794 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5df92110 lockFactory=org.apache.lucene.store.NativeFSLockFactory@73d5786f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 716285 T1794 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 716289 T1794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 716289 T1794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 716290 T1794 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 716291 T1794 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 716291 T1794 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 716292 T1794 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 716292 T1794 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 716293 T1794 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 716293 T1794 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 716307 T1794 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 716318 T1794 oass.SolrIndexSearcher.<init> Opening Searcher@3d547187 main
[junit4:junit4] 2> 716319 T1794 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/control/data/tlog
[junit4:junit4] 2> 716320 T1794 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 716320 T1794 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 716325 T1795 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d547187 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 716326 T1794 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 716327 T1794 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49645/e_ro/m collection:control_collection shard:shard1
[junit4:junit4] 2> 716333 T1794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 716352 T1794 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 716358 T1794 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 716359 T1794 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 716359 T1794 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49645/e_ro/m/collection1/
[junit4:junit4] 2> 716359 T1794 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 716359 T1794 oasc.SyncStrategy.syncToMe http://127.0.0.1:49645/e_ro/m/collection1/ has no replicas
[junit4:junit4] 2> 716360 T1794 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49645/e_ro/m/collection1/
[junit4:junit4] 2> 716360 T1794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 717713 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 717732 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 717761 T1794 oasc.ZkController.register We are http://127.0.0.1:49645/e_ro/m/collection1/ and leader is http://127.0.0.1:49645/e_ro/m/collection1/
[junit4:junit4] 2> 717762 T1794 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49645/e_ro/m
[junit4:junit4] 2> 717762 T1794 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 717762 T1794 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 717763 T1794 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 717777 T1794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 717779 T1769 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 717780 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 717780 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 717805 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 717808 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 717812 T1798 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64bd8254 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 717812 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 717816 T1769 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 717824 T1769 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 718239 T1769 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 718244 T1769 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49649
[junit4:junit4] 2> 718245 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 718245 T1769 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 718245 T1769 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669
[junit4:junit4] 2> 718246 T1769 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/solr.xml
[junit4:junit4] 2> 718246 T1769 oasc.CoreContainer.<init> New CoreContainer 164202453
[junit4:junit4] 2> 718247 T1769 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/'
[junit4:junit4] 2> 718247 T1769 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/'
[junit4:junit4] 2> 718401 T1769 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 718402 T1769 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 718403 T1769 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 718403 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 718403 T1769 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 718404 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 718404 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 718405 T1769 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 718405 T1769 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 718406 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 718426 T1769 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 718426 T1769 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49642/solr
[junit4:junit4] 2> 718427 T1769 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 718429 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 718434 T1809 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8d1640f name:ZooKeeperConnection Watcher:127.0.0.1:49642 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 718435 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 718441 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 718485 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 718489 T1811 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c153530 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 718489 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 718500 T1769 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 719253 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 719255 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49645_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49645/e_ro/m"}
[junit4:junit4] 2> 719269 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 719269 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 719269 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 719511 T1769 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49649_e_ro%2Fm
[junit4:junit4] 2> 719514 T1769 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49649_e_ro%2Fm
[junit4:junit4] 2> 719519 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 719521 T1798 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 719522 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 719522 T1791 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 719523 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 719528 T1811 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 719537 T1812 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 719537 T1812 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 720783 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 720785 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49649_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49649/e_ro/m"}
[junit4:junit4] 2> 720785 T1792 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 720785 T1792 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 720796 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 720805 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 720805 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 721552 T1812 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/collection1
[junit4:junit4] 2> 721552 T1812 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 721554 T1812 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 721554 T1812 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 721558 T1812 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/collection1/'
[junit4:junit4] 2> 721559 T1812 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 721560 T1812 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/collection1/lib/README' to classloader
[junit4:junit4] 2> 721628 T1812 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 721717 T1812 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 721721 T1812 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 721728 T1812 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 722549 T1812 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 722562 T1812 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 722566 T1812 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 722638 T1812 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 722648 T1812 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 722656 T1812 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 722658 T1812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 722659 T1812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 722660 T1812 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 722662 T1812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 722662 T1812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 722663 T1812 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 722663 T1812 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1372203317669/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1/
[junit4:junit4] 2> 722663 T1812 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ec40b92
[junit4:junit4] 2> 722664 T1812 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 722665 T1812 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1
[junit4:junit4] 2> 722665 T1812 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1/index/
[junit4:junit4] 2> 722665 T1812 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 722666 T1812 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1/index
[junit4:junit4] 2> 722670 T1812 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b948d26 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7734f6ba),segFN=segments_1,generation=1}
[junit4:junit4] 2> 722670 T1812 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 722680 T1812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 722680 T1812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 722681 T1812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 722682 T1812 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 722683 T1812 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 722683 T1812 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 722683 T1812 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 722684 T1812 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 722685 T1812 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 722700 T1812 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 722706 T1812 oass.SolrIndexSearcher.<init> Opening Searcher@2b4305b8 main
[junit4:junit4] 2> 722707 T1812 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty1/tlog
[junit4:junit4] 2> 722708 T1812 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 722709 T1812 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 722715 T1813 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b4305b8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 722718 T1812 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 722718 T1812 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49649/e_ro/m collection:collection1 shard:shard1
[junit4:junit4] 2> 722720 T1812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 722742 T1812 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 722748 T1812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 722748 T1812 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 722750 T1812 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49649/e_ro/m/collection1/
[junit4:junit4] 2> 722750 T1812 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 722751 T1812 oasc.SyncStrategy.syncToMe http://127.0.0.1:49649/e_ro/m/collection1/ has no replicas
[junit4:junit4] 2> 722751 T1812 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49649/e_ro/m/collection1/
[junit4:junit4] 2> 722751 T1812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 723826 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 723860 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 723873 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 723875 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 723903 T1812 oasc.ZkController.register We are http://127.0.0.1:49649/e_ro/m/collection1/ and leader is http://127.0.0.1:49649/e_ro/m/collection1/
[junit4:junit4] 2> 723904 T1812 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49649/e_ro/m
[junit4:junit4] 2> 723904 T1812 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 723904 T1812 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 723904 T1812 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 723909 T1812 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 723912 T1769 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 723913 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 723913 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 724349 T1769 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 724355 T1769 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49652
[junit4:junit4] 2> 724356 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 724357 T1769 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 724357 T1769 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781
[junit4:junit4] 2> 724358 T1769 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/solr.xml
[junit4:junit4] 2> 724358 T1769 oasc.CoreContainer.<init> New CoreContainer 2123053172
[junit4:junit4] 2> 724359 T1769 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/'
[junit4:junit4] 2> 724359 T1769 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/'
[junit4:junit4] 2> 724525 T1769 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 724526 T1769 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 724526 T1769 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 724527 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 724527 T1769 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 724528 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 724528 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 724529 T1769 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 724529 T1769 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 724530 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 724552 T1769 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 724553 T1769 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49642/solr
[junit4:junit4] 2> 724553 T1769 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 724555 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 724559 T1825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77c1d334 name:ZooKeeperConnection Watcher:127.0.0.1:49642 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 724560 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 724565 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 724582 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 724587 T1827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76ddd7c4 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 724587 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 724597 T1769 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 725377 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 725378 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49649_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49649/e_ro/m"}
[junit4:junit4] 2> 725410 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725410 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725410 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725413 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725613 T1769 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49652_e_ro%2Fm
[junit4:junit4] 2> 725619 T1769 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49652_e_ro%2Fm
[junit4:junit4] 2> 725634 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725641 T1798 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 725643 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 725644 T1791 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 725644 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 725647 T1827 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 725648 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 725654 T1811 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 725666 T1828 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 725666 T1828 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 726937 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 726938 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49652_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49652/e_ro/m"}
[junit4:junit4] 2> 726939 T1792 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 726939 T1792 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 726950 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 726950 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 726954 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 726955 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727671 T1828 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/collection1
[junit4:junit4] 2> 727672 T1828 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 727673 T1828 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 727674 T1828 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 727677 T1828 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/collection1/'
[junit4:junit4] 2> 727678 T1828 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 727679 T1828 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/collection1/lib/README' to classloader
[junit4:junit4] 2> 727761 T1828 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 727849 T1828 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 727853 T1828 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 727862 T1828 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 728688 T1828 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 728699 T1828 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 728708 T1828 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 728741 T1828 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 728748 T1828 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 728756 T1828 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 728759 T1828 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 728759 T1828 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 728760 T1828 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 728762 T1828 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 728763 T1828 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 728763 T1828 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 728764 T1828 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/
[junit4:junit4] 2> 728764 T1828 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ec40b92
[junit4:junit4] 2> 728764 T1828 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 728765 T1828 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2
[junit4:junit4] 2> 728765 T1828 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index/
[junit4:junit4] 2> 728766 T1828 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 728766 T1828 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index
[junit4:junit4] 2> 728770 T1828 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 728770 T1828 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 728774 T1828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 728775 T1828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 728776 T1828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 728781 T1828 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 728783 T1828 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 728783 T1828 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 728783 T1828 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 728784 T1828 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 728785 T1828 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 728798 T1828 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 728806 T1828 oass.SolrIndexSearcher.<init> Opening Searcher@50c583c main
[junit4:junit4] 2> 728806 T1828 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/tlog
[junit4:junit4] 2> 728808 T1828 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 728808 T1828 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 728815 T1829 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50c583c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 728818 T1828 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 728819 T1828 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49652/e_ro/m collection:collection1 shard:shard1
[junit4:junit4] 2> 728826 T1828 oasc.ZkController.register We are http://127.0.0.1:49652/e_ro/m/collection1/ and leader is http://127.0.0.1:49649/e_ro/m/collection1/
[junit4:junit4] 2> 728826 T1828 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49652/e_ro/m
[junit4:junit4] 2> 728826 T1828 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 728827 T1828 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1581 name=collection1 org.apache.solr.core.SolrCore@53198fc9 url=http://127.0.0.1:49652/e_ro/m/collection1 node=127.0.0.1:49652_e_ro%2Fm C1581_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49652_e_ro%2Fm, base_url=http://127.0.0.1:49652/e_ro/m}
[junit4:junit4] 2> 728828 T1830 C1581 P49652 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 728829 T1828 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 728830 T1830 C1581 P49652 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 728830 T1830 C1581 P49652 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 728831 T1830 C1581 P49652 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 728834 T1769 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 728834 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 728834 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 728837 T1830 C1581 P49652 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 728859 T1769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 728862 T1803 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> ASYNC NEW_CORE C1582 name=collection1 org.apache.solr.core.SolrCore@1a7ce791 url=http://127.0.0.1:49645/e_ro/m/collection1 node=127.0.0.1:49645_e_ro%2Fm C1582_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49645_e_ro%2Fm, base_url=http://127.0.0.1:49645/e_ro/m, leader=true}
[junit4:junit4] 2> 728881 T1786 C1582 P49645 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5df92110 lockFactory=org.apache.lucene.store.NativeFSLockFactory@73d5786f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 728882 T1786 C1582 P49645 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 728886 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10001 (1438859477619048448)]} 0 8
[junit4:junit4] 2> 728886 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[1 (1438859477620097024)]} 0 8
[junit4:junit4] 2> ASYNC NEW_CORE C1583 name=collection1 org.apache.solr.core.SolrCore@7bf91eea url=http://127.0.0.1:49649/e_ro/m/collection1 node=127.0.0.1:49649_e_ro%2Fm C1583_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49649_e_ro%2Fm, base_url=http://127.0.0.1:49649/e_ro/m, leader=true}
[junit4:junit4] 2> 728916 T1805 C1583 P49649 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b948d26 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7734f6ba),segFN=segments_1,generation=1}
[junit4:junit4] 2> 728918 T1805 C1583 P49649 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 728921 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10001 (1438859477655748608)]} 0 8
[junit4:junit4] 2> 728922 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[1 (1438859477655748609)]} 0 9
[junit4:junit4] 2> 728955 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10002 (1438859477697691648)]} 0 2
[junit4:junit4] 2> 728963 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[2 (1438859477701885952)]} 0 8
[junit4:junit4] 2> 728968 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10002 (1438859477711323136)]} 0 2
[junit4:junit4] 2> 728969 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[2 (1438859477713420288)]} 0 1
[junit4:junit4] 2> 729072 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10003 (1438859477820375040)]} 0 2
[junit4:junit4] 2> 729073 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[3 (1438859477821423616)]} 0 3
[junit4:junit4] 2> 729077 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10003 (1438859477825617920)]} 0 2
[junit4:junit4] 2> 729078 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[3 (1438859477826666496)]} 0 2
[junit4:junit4] 2> 729111 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10001 (-1438859477863366656)]} 0 1
[junit4:junit4] 2> 729112 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[1 (-1438859477863366657)]} 0 1
[junit4:junit4] 2> 729116 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10001 (-1438859477868609536)]} 0 1
[junit4:junit4] 2> 729118 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[1 (-1438859477869658112)]} 0 1
[junit4:junit4] 2> 729122 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10004 (1438859477872803840)]} 0 2
[junit4:junit4] 2> 729123 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[4 (1438859477874900992)]} 0 1
[junit4:junit4] 2> 729126 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10004 (1438859477878046720)]} 0 1
[junit4:junit4] 2> 729128 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[4 (1438859477880143872)]} 0 1
[junit4:junit4] 2> 729208 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[2 (-1438859477965078528)]} 0 0
[junit4:junit4] 2> 729208 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10002 (-1438859477965078529)]} 0 0
[junit4:junit4] 2> 729217 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[2 (-1438859477973467136)]} 0 1
[junit4:junit4] 2> 729218 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10002 (-1438859477974515712)]} 0 1
[junit4:junit4] 2> 729224 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10005 (1438859477980807168)]} 0 1
[junit4:junit4] 2> 729226 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[5 (1438859477980807169)]} 0 3
[junit4:junit4] 2> 729229 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10005 (1438859477986050048)]} 0 2
[junit4:junit4] 2> 729230 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[5 (1438859477987098624)]} 0 1
[junit4:junit4] 2> 729241 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10003 (-1438859477999681536)]} 0 1
[junit4:junit4] 2> 729242 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[3 (-1438859478000730112)]} 0 0
[junit4:junit4] 2> 729246 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10003 (-1438859478004924416)]} 0 1
[junit4:junit4] 2> 729248 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[3 (-1438859478007021568)]} 0 1
[junit4:junit4] 2> 729251 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10006 (1438859478009118720)]} 0 1
[junit4:junit4] 2> 729254 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[6 (1438859478012264448)]} 0 1
[junit4:junit4] 2> 729257 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10006 (1438859478015410176)]} 0 1
[junit4:junit4] 2> 729260 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[6 (1438859478018555904)]} 0 1
[junit4:junit4] 2> 729312 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10004 (-1438859478074130432)]} 0 0
[junit4:junit4] 2> 729314 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[4 (-1438859478076227584)]} 0 0
[junit4:junit4] 2> 729319 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10004 (-1438859478080421888)]} 0 1
[junit4:junit4] 2> 729320 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[4 (-1438859478082519040)]} 0 1
[junit4:junit4] 2> 729325 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10007 (1438859478086713344)]} 0 2
[junit4:junit4] 2> 729327 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[7 (1438859478088810496)]} 0 1
[junit4:junit4] 2> 729336 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10007 (1438859478095101952)]} 0 5
[junit4:junit4] 2> 729340 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[7 (1438859478101393408)]} 0 2
[junit4:junit4] 2> 729381 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10005 (-1438859478145433600)]} 0 1
[junit4:junit4] 2> 729381 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[5 (-1438859478146482176)]} 0 1
[junit4:junit4] 2> 729390 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10005 (-1438859478155919360)]} 0 0
[junit4:junit4] 2> 729393 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[5 (-1438859478159065088)]} 0 0
[junit4:junit4] 2> 729397 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10008 (1438859478161162240)]} 0 2
[junit4:junit4] 2> 729400 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[8 (1438859478165356544)]} 0 1
[junit4:junit4] 2> 729404 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10008 (1438859478168502272)]} 0 2
[junit4:junit4] 2> 729407 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[8 (1438859478171648000)]} 0 2
[junit4:junit4] 2> 729450 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10009 (1438859478214639616)]} 0 5
[junit4:junit4] 2> 729452 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[9 (1438859478218833920)]} 0 2
[junit4:junit4] 2> 729455 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10009 (1438859478223028224)]} 0 1
[junit4:junit4] 2> 729456 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[9 (1438859478224076800)]} 0 1
[junit4:junit4] 2> 729497 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10010 (1438859478264971264)]} 0 4
[junit4:junit4] 2> 729499 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10 (1438859478266019840)]} 0 5
[junit4:junit4] 2> 729509 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10010 (1438859478277554176)]} 0 3
[junit4:junit4] 2> 729512 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10 (1438859478281748480)]} 0 2
[junit4:junit4] 2> 729534 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10006 (-1438859478306914304)]} 0 0
[junit4:junit4] 2> 729535 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[6 (-1438859478307962880)]} 0 0
[junit4:junit4] 2> 729538 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10006 (-1438859478310060032)]} 0 1
[junit4:junit4] 2> 729539 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[6 (-1438859478311108608)]} 0 1
[junit4:junit4] 2> 729542 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10011 (1438859478314254336)]} 0 2
[junit4:junit4] 2> 729546 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[11 (1438859478317400064)]} 0 1
[junit4:junit4] 2> 729548 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10011 (1438859478319497216)]} 0 2
[junit4:junit4] 2> 729551 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[11 (1438859478323691520)]} 0 2
[junit4:junit4] 2> 729652 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10012 (1438859478428549120)]} 0 3
[junit4:junit4] 2> 729654 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[12 (1438859478431694848)]} 0 1
[junit4:junit4] 2> 729659 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[12 (1438859478436937728)]} 0 2
[junit4:junit4] 2> 729660 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10012 (1438859478436937729)]} 0 3
[junit4:junit4] 2> 729738 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[13 (1438859478516629504)]} 0 6
[junit4:junit4] 2> 729742 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10013 (1438859478523969536)]} 0 2
[junit4:junit4] 2> 729744 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[13 (1438859478525018112)]} 0 2
[junit4:junit4] 2> 729747 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10013 (1438859478529212416)]} 0 2
[junit4:junit4] 2> 729804 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[7 (-1438859478588981248)]} 0 1
[junit4:junit4] 2> 729809 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10007 (-1438859478594224128)]} 0 2
[junit4:junit4] 2> 729810 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[7 (-1438859478596321280)]} 0 0
[junit4:junit4] 2> 729820 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10007 (-1438859478606807040)]} 0 1
[junit4:junit4] 2> 729823 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[14 (1438859478607855616)]} 0 2
[junit4:junit4] 2> 729825 T1783 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10014 (1438859478611001344)]} 0 1
[junit4:junit4] 2> 729832 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[14 (1438859478618341376)]} 0 2
[junit4:junit4] 2> 729833 T1801 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10014 (1438859478618341377)]} 0 3
[junit4:junit4] 2> 729879 T1787 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[15 (1438859478666575872)]} 0 2
[junit4:junit4] 2> 729880 T1784 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10015 (1438859478667624448)]} 0 3
[junit4:junit4] 2> 729885 T1804 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10015 (1438859478673915904)]} 0 1
[junit4:junit4] 2> 729886 T1805 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[15 (1438859478673915905)]} 0 2
[junit4:junit4] 2> 729899 T1782 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10008 (-1438859478688595968)]} 0 1
[junit4:junit4] 2> 729900 T1785 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[8 (-1438859478689644544)]} 0 1
[junit4:junit4] 2> 729902 T1806 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10008 (-1438859478692790272)]} 0 0
[junit4:junit4] 2> 729906 T1802 C1583 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[8 (-1438859478696984576)]} 0 1
[junit4:junit4] 2> 729911 T1786 C1582 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10016 (1438859478700130304)]} 0 2
[junit4:junit4] 2> ASYNC NEW_CORE C1584 name=collection1 org.apache.solr.core.SolrCore@1a7ce791 url=http://127.0.0.1:49645/e_ro/m/collection1 node=127.0.0.1:49645_e_ro%2Fm C1584_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49645_e_ro%2Fm, base_url=http://127.0.0.1:49645/e_ro/m, leader=true}
[junit4:junit4] 2> 729912 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[16 (1438859478702227456)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C1585 name=collection1 org.apache.solr.core.SolrCore@7bf91eea url=http://127.0.0.1:49649/e_ro/m/collection1 node=127.0.0.1:49649_e_ro%2Fm C1585_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49649_e_ro%2Fm, base_url=http://127.0.0.1:49649/e_ro/m, leader=true}
[junit4:junit4] 2> 729928 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10016 (1438859478717956096)]} 0 2
[junit4:junit4] 2> 729930 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[16 (1438859478721101824)]} 0 2
[junit4:junit4] 2> 729969 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10009 (-1438859478763044864)]} 0 1
[junit4:junit4] 2> 729971 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[9 (-1438859478765142016)]} 0 1
[junit4:junit4] 2> 729973 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10009 (-1438859478767239168)]} 0 1
[junit4:junit4] 2> 729975 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[9 (-1438859478769336320)]} 0 0
[junit4:junit4] 2> 729978 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10017 (1438859478770384896)]} 0 2
[junit4:junit4] 2> 729985 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[17 (1438859478777724928)]} 0 2
[junit4:junit4] 2> 729987 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10017 (1438859478780870656)]} 0 1
[junit4:junit4] 2> 729989 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 729991 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49652_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49652/e_ro/m"}
[junit4:junit4] 2> 729993 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[17 (1438859478787162112)]} 0 2
[junit4:junit4] 2> 730008 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 730008 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 730009 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 730009 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10010 (-1438859478803939328)]} 0 1
[junit4:junit4] 2> 730009 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 730019 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10 (-1438859478815473664)]} 0 0
[junit4:junit4] 2> ASYNC NEW_CORE C1586 name=collection1 org.apache.solr.core.SolrCore@53198fc9 url=http://127.0.0.1:49652/e_ro/m/collection1 node=127.0.0.1:49652_e_ro%2Fm C1586_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49652_e_ro%2Fm, base_url=http://127.0.0.1:49652/e_ro/m}
[junit4:junit4] 2> 730038 T1820 C1586 P49652 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 730040 T1820 C1586 P49652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 730041 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10010 (-1438859478817570816)]} 0 7
[junit4:junit4] 2> 730043 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10010 (-1438859478817570816)]} 0 22
[junit4:junit4] 2> 730043 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10 (-1438859478822813696)]} 0 6
[junit4:junit4] 2> 730044 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10 (-1438859478822813696)]} 0 18
[junit4:junit4] 2> 730049 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[18 (1438859478844833792)]} 0 2
[junit4:junit4] 2> 730049 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10018 (1438859478844833793)]} 0 2
[junit4:junit4] 2> 730058 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438859478850076672)]} 0 2
[junit4:junit4] 2> 730060 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[18 (1438859478850076672)]} 0 8
[junit4:junit4] 2> 730064 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10018 (1438859478852173824)]} 0 3
[junit4:junit4] 2> 730065 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10018 (1438859478852173824)]} 0 11
[junit4:junit4] 2> 730148 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[12 (-1438859478950739968)]} 0 1
[junit4:junit4] 2> 730151 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10012 (-1438859478953885696)]} 0 0
[junit4:junit4] 2> 730156 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[12 (-1438859478955982848)]} 0 0
[junit4:junit4] 2> 730158 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10012 (-1438859478957031424)]} 0 1
[junit4:junit4] 2> 730158 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[12 (-1438859478955982848)]} 0 5
[junit4:junit4] 2> 730159 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10012 (-1438859478957031424)]} 0 5
[junit4:junit4] 2> 730163 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[19 (1438859478965420032)]} 0 2
[junit4:junit4] 2> 730164 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10019 (1438859478965420033)]} 0 2
[junit4:junit4] 2> 730172 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438859478970662912)]} 0 2
[junit4:junit4] 2> 730173 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10019 (1438859478971711488)]} 0 1
[junit4:junit4] 2> 730175 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[19 (1438859478970662912)]} 0 9
[junit4:junit4] 2> 730183 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10019 (1438859478971711488)]} 0 15
[junit4:junit4] 2> 730231 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[15 (-1438859479036723200)]} 0 1
[junit4:junit4] 2> 730238 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10015 (-1438859479045111808)]} 0 0
[junit4:junit4] 2> 730241 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[15 (-1438859479043014656)]} 0 0
[junit4:junit4] 2> 730243 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[15 (-1438859479043014656)]} 0 7
[junit4:junit4] 2> 730253 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10015 (-1438859479051403264)]} 0 0
[junit4:junit4] 2> 730256 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[20 (1438859479061889024)]} 0 2
[junit4:junit4] 2> 730257 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10015 (-1438859479051403264)]} 0 13
[junit4:junit4] 2> 730263 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10020 (1438859479070277632)]} 0 1
[junit4:junit4] 2> 730265 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438859479068180480)]} 0 1
[junit4:junit4] 2> 730266 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[20 (1438859479068180480)]} 0 7
[junit4:junit4] 2> 730273 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10020 (1438859479076569088)]} 0 1
[junit4:junit4] 2> 730274 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10020 (1438859479076569088)]} 0 6
[junit4:junit4] 2> 730283 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[21 (1438859479091249152)]} 0 1
[junit4:junit4] 2> 730291 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10021 (1438859479099637760)]} 0 2
[junit4:junit4] 2> 730293 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438859479096492032)]} 0 1
[junit4:junit4] 2> 730294 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[21 (1438859479096492032)]} 0 7
[junit4:junit4] 2> 730302 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10021 (1438859479105929216)]} 0 2
[junit4:junit4] 2> 730303 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10021 (1438859479105929216)]} 0 7
[junit4:junit4] 2> 730329 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[22 (1438859479137386496)]} 0 3
[junit4:junit4] 2> 730335 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10022 (1438859479145775104)]} 0 1
[junit4:junit4] 2> 730354 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438859479159406592)]} 0 2
[junit4:junit4] 2> 730356 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[22 (1438859479159406592)]} 0 10
[junit4:junit4] 2> 730358 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10022 (1438859479161503744)]} 0 1
[junit4:junit4] 2> 730359 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10022 (1438859479161503744)]} 0 10
[junit4:junit4] 2> 730368 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[23 (1438859479180378112)]} 0 2
[junit4:junit4] 2> 730377 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10023 (1438859479189815296)]} 0 1
[junit4:junit4] 2> 730381 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438859479186669568)]} 0 1
[junit4:junit4] 2> 730385 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[23 (1438859479186669568)]} 0 13
[junit4:junit4] 2> 730389 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10023 (1438859479197155328)]} 0 2
[junit4:junit4] 2> 730390 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10023 (1438859479197155328)]} 0 7
[junit4:junit4] 2> 730486 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[24 (1438859479303061504)]} 0 2
[junit4:junit4] 2> 730488 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10024 (1438859479303061505)]} 0 4
[junit4:junit4] 2> 730499 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10024 (1438859479312498688)]} 0 1
[junit4:junit4] 2> 730502 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438859479315644416)]} 0 2
[junit4:junit4] 2> 730506 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[24 (1438859479315644416)]} 0 10
[junit4:junit4] 2> 730507 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10024 (1438859479312498688)]} 0 15
[junit4:junit4] 2> 730583 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10016 (-1438859479406870528)]} 0 1
[junit4:junit4] 2> 730584 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[16 (-1438859479407919104)]} 0 0
[junit4:junit4] 2> 730597 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10016 (-1438859479418404864)]} 0 0
[junit4:junit4] 2> 730602 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[16 (-1438859479418404865)]} 0 1
[junit4:junit4] 2> 730606 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10016 (-1438859479418404864)]} 0 12
[junit4:junit4] 2> 730609 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[16 (-1438859479418404865)]} 0 15
[junit4:junit4] 2> 730615 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10025 (1438859479439376384)]} 0 2
[junit4:junit4] 2> 730616 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[25 (1438859479439376385)]} 0 2
[junit4:junit4] 2> 730624 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10025 (1438859479444619264)]} 0 2
[junit4:junit4] 2> 730625 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10025 (1438859479444619264)]} 0 7
[junit4:junit4] 2> 730625 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1438859479445667840)]} 0 2
[junit4:junit4] 2> 730626 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[25 (1438859479445667840)]} 0 6
[junit4:junit4] 2> 730669 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10017 (-1438859479497048064)]} 0 1
[junit4:junit4] 2> 730669 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[17 (-1438859479497048065)]} 0 0
[junit4:junit4] 2> 730679 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[17 (-1438859479503339520)]} 0 0
[junit4:junit4] 2> 730680 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10017 (-1438859479502290944)]} 0 0
[junit4:junit4] 2> 730681 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[17 (-1438859479503339520)]} 0 7
[junit4:junit4] 2> 730681 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10017 (-1438859479502290944)]} 0 7
[junit4:junit4] 2> 730686 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[26 (1438859479513825280)]} 0 2
[junit4:junit4] 2> 730687 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10026 (1438859479513825281)]} 0 2
[junit4:junit4] 2> 730694 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1438859479518019584)]} 0 1
[junit4:junit4] 2> 730695 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[26 (1438859479518019584)]} 0 6
[junit4:junit4] 2> 730696 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10026 (1438859479519068160)]} 0 2
[junit4:junit4] 2> 730697 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10026 (1438859479519068160)]} 0 7
[junit4:junit4] 2> 730796 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[27 (1438859479628120064)]} 0 2
[junit4:junit4] 2> 730804 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10027 (1438859479636508672)]} 0 3
[junit4:junit4] 2> 730810 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1438859479638605824)]} 0 1
[junit4:junit4] 2> 730812 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[27 (1438859479638605824)]} 0 8
[junit4:junit4] 2> 730812 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10027 (1438859479642800128)]} 0 1
[junit4:junit4] 2> 730813 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10027 (1438859479642800128)]} 0 5
[junit4:junit4] 2> 730842 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[18 (-1438859479677403136)]} 0 1
[junit4:junit4] 2> 730843 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10018 (-1438859479679500288)]} 0 0
[junit4:junit4] 2> 730854 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[18 (-1438859479685791744)]} 0 0
[junit4:junit4] 2> 730855 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10018 (-1438859479687888896)]} 0 0
[junit4:junit4] 2> 730856 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10018 (-1438859479687888896)]} 0 5
[junit4:junit4] 2> 730857 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[18 (-1438859479685791744)]} 0 8
[junit4:junit4] 2> 730860 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10028 (1438859479696277504)]} 0 1
[junit4:junit4] 2> 730861 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[28 (1438859479697326080)]} 0 1
[junit4:junit4] 2> 730865 T1803 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 730866 T1803 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:49652_e_ro%252Fm&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 730868 T1769 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49652
[junit4:junit4] 2> 730869 T1769 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2123053172
[junit4:junit4] 2> 730875 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10028 (1438859479701520384)]} 0 2
[junit4:junit4] 2> 730876 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10028 (1438859479701520384)]} 0 12
[junit4:junit4] 2> 730876 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1438859479704666112)]} 0 3
[junit4:junit4] 2> 730877 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[28 (1438859479704666112)]} 0 10
[junit4:junit4] 2> 730890 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10029 (1438859479727734784)]} 0 2
[junit4:junit4] 2> 730891 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[29 (1438859479727734785)]} 0 2
[junit4:junit4] 2> 730898 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10029 (1438859479732977664)]} 0 1
[junit4:junit4] 2> 730899 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1438859479732977665)]} 0 1
[junit4:junit4] 2> 730900 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10029 (1438859479732977664)]} 0 7
[junit4:junit4] 2> 730901 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[29 (1438859479732977665)]} 0 7
[junit4:junit4] 2> 730975 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10020 (-1438859479817912320)]} 0 0
[junit4:junit4] 2> 730978 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[20 (-1438859479820009472)]} 0 1
[junit4:junit4] 2> 730983 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10020 (-1438859479823155200)]} 0 1
[junit4:junit4] 2> 730985 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[20 (-1438859479825252352)]} 0 1
[junit4:junit4] 2> 730985 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10020 (-1438859479823155200)]} 0 5
[junit4:junit4] 2> 730986 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[20 (-1438859479825252352)]} 0 5
[junit4:junit4] 2> 730990 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10030 (1438859479832592384)]} 0 1
[junit4:junit4] 2> 730991 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[30 (1438859479832592385)]} 0 2
[junit4:junit4] 2> 730998 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10030 (1438859479836786688)]} 0 1
[junit4:junit4] 2> 731000 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1438859479837835264)]} 0 1
[junit4:junit4] 2> 731001 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10030 (1438859479836786688)]} 0 8
[junit4:junit4] 2> 731001 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[30 (1438859479837835264)]} 0 7
[junit4:junit4] 2> 731030 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10027 (-1438859479875584000)]} 0 1
[junit4:junit4] 2> 731033 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[27 (-1438859479878729728)]} 0 1
[junit4:junit4] 2> 731048 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10027 (-1438859479888166912)]} 0 1
[junit4:junit4] 2> 731049 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[27 (-1438859479888166913)]} 0 1
[junit4:junit4] 2> 731051 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10027 (-1438859479888166912)]} 0 9
[junit4:junit4] 2> 731052 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[27 (-1438859479888166913)]} 0 10
[junit4:junit4] 2> 731056 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10031 (1438859479901798400)]} 0 2
[junit4:junit4] 2> 731057 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[31 (1438859479901798401)]} 0 2
[junit4:junit4] 2> 731066 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10031 (1438859479905992704)]} 0 3
[junit4:junit4] 2> 731067 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10031 (1438859479905992704)]} 0 8
[junit4:junit4] 2> 731069 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1438859479907041280)]} 0 2
[junit4:junit4] 2> 731072 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[31 (1438859479907041280)]} 0 12
[junit4:junit4] 2> 731113 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10031 (-1438859479962615808)]} 0 0
[junit4:junit4] 2> 731118 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[31 (-1438859479966810112)]} 0 1
[junit4:junit4] 2> 731120 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10031 (-1438859479966810112)]} 0 0
[junit4:junit4] 2> 731121 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10031 (-1438859479966810112)]} 0 4
[junit4:junit4] 2> 731124 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[31 (-1438859479971004416)]} 0 1
[junit4:junit4] 2> 731125 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[31 (-1438859479971004416)]} 0 5
[junit4:junit4] 2> 731125 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10032 (1438859479974150144)]} 0 1
[junit4:junit4] 2> 731131 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[32 (1438859479980441600)]} 0 2
[junit4:junit4] 2> 731136 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10032 (1438859479981490176)]} 0 1
[junit4:junit4] 2> 731137 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10032 (1438859479981490176)]} 0 7
[junit4:junit4] 2> 731140 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1438859479985684480)]} 0 2
[junit4:junit4] 2> 731141 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[32 (1438859479985684480)]} 0 6
[junit4:junit4] 2> 731190 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10033 (1438859480041259008)]} 0 2
[junit4:junit4] 2> 731191 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[33 (1438859480041259009)]} 0 3
[junit4:junit4] 2> 731211 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10033 (1438859480049647616)]} 0 4
[junit4:junit4] 2> 731212 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10033 (1438859480049647616)]} 0 16
[junit4:junit4] 2> 731212 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1438859480056987648)]} 0 1
[junit4:junit4] 2> 731214 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[33 (1438859480056987648)]} 0 12
[junit4:junit4] 2> 731293 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10033 (-1438859480151359488)]} 0 0
[junit4:junit4] 2> 731294 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[33 (-1438859480152408064)]} 0 0
[junit4:junit4] 2> 731305 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10033 (-1438859480160796672)]} 0 1
[junit4:junit4] 2> 731305 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[33 (-1438859480160796673)]} 0 0
[junit4:junit4] 2> 731306 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10033 (-1438859480160796672)]} 0 4
[junit4:junit4] 2> 731307 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[33 (-1438859480160796673)]} 0 5
[junit4:junit4] 2> 731311 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10034 (1438859480168136704)]} 0 2
[junit4:junit4] 2> 731312 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[34 (1438859480170233856)]} 0 2
[junit4:junit4] 2> 731319 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10034 (1438859480173379584)]} 0 1
[junit4:junit4] 2> 731320 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1438859480174428160)]} 0 2
[junit4:junit4] 2> 731320 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10034 (1438859480173379584)]} 0 6
[junit4:junit4] 2> 731321 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[34 (1438859480174428160)]} 0 6
[junit4:junit4] 2> 731378 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10034 (-1438859480239439872)]} 0 2
[junit4:junit4] 2> 731378 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[34 (-1438859480240488448)]} 0 0
[junit4:junit4] 2> 731384 T1823 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10034 (-1438859480244682752)]} 0 0
[junit4:junit4] 2> 731386 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10034 (-1438859480244682752)]} 0 4
[junit4:junit4] 2> 731386 T1819 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[34 (-1438859480244682753)]} 0 1
[junit4:junit4] 2> 731387 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[34 (-1438859480244682753)]} 0 5
[junit4:junit4] 2> 731391 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10035 (1438859480253071360)]} 0 2
[junit4:junit4] 2> 731392 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[35 (1438859480254119936)]} 0 1
[junit4:junit4] 2> 731405 T1820 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10035 (1438859480257265664)]} 0 2
[junit4:junit4] 2> 731406 T1818 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1438859480264605696)]} 0 2
[junit4:junit4] 2> 731406 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10035 (1438859480257265664)]} 0 12
[junit4:junit4] 2> 731407 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[35 (1438859480264605696)]} 0 6
[junit4:junit4] 2> 731466 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10036 (1438859480331714560)]} 0 2
[junit4:junit4] 2> 731467 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[36 (1438859480331714561)]} 0 2
[junit4:junit4] 2> 731474 T1821 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10036 (1438859480336957440)]} 0 1
[junit4:junit4] 2> 731475 T1822 C1586 P49652 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={distrib.from=http://127.0.0.1:49649/e_ro/m/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1438859480336957441)]} 0 1
[junit4:junit4] 2> 731475 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10036 (1438859480336957440)]} 0 6
[junit4:junit4] 2> 731476 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[36 (1438859480336957441)]} 0 6
[junit4:junit4] 2> 731533 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 731535 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:49652_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49652/e_ro/m"}
[junit4:junit4] 2> 731545 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 731546 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 731547 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 731547 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 731581 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[37 (1438859480451252224)]} 0 2
[junit4:junit4] 2> 731583 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10037 (1438859480452300800)]} 0 4
[junit4:junit4] 2> 731591 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[37 (1438859480461737984)]} 0 2
[junit4:junit4] 2> 731592 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10037 (1438859480463835136)]} 0 2
[junit4:junit4] 2> 731695 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[35 (-1438859480572887040)]} 0 0
[junit4:junit4] 2> 731697 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10035 (-1438859480574984192)]} 0 0
[junit4:junit4] 2> 731701 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[35 (-1438859480578129920)]} 0 1
[junit4:junit4] 2> 731702 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10035 (-1438859480579178496)]} 0 1
[junit4:junit4] 2> 731709 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[38 (1438859480585469952)]} 0 3
[junit4:junit4] 2> 731713 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10038 (1438859480590712832)]} 0 1
[junit4:junit4] 2> 731717 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[38 (1438859480595955712)]} 0 1
[junit4:junit4] 2> 731720 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10038 (1438859480598052864)]} 0 1
[junit4:junit4] 2> 731781 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[39 (1438859480662016000)]} 0 2
[junit4:junit4] 2> 731786 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10039 (1438859480667258880)]} 0 1
[junit4:junit4] 2> 731790 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[39 (1438859480671453184)]} 0 1
[junit4:junit4] 2> 731793 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10039 (1438859480674598912)]} 0 1
[junit4:junit4] 2> 731844 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[40 (1438859480724930560)]} 0 6
[junit4:junit4] 2> 731846 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10040 (1438859480725979136)]} 0 6
[junit4:junit4] 2> 731852 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[40 (1438859480736464896)]} 0 1
[junit4:junit4] 2> 731854 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10040 (1438859480738562048)]} 0 1
[junit4:junit4] 2> 731878 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10039 (-1438859480764776448)]} 0 0
[junit4:junit4] 2> 731878 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[39 (-1438859480764776449)]} 0 0
[junit4:junit4] 2> 731882 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10039 (-1438859480767922176)]} 0 1
[junit4:junit4] 2> 731882 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[39 (-1438859480768970752)]} 0 0
[junit4:junit4] 2> 731886 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10041 (1438859480772116480)]} 0 1
[junit4:junit4] 2> 731887 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[41 (1438859480773165056)]} 0 2
[junit4:junit4] 2> 731890 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10041 (1438859480776310784)]} 0 1
[junit4:junit4] 2> 731895 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[41 (1438859480778407936)]} 0 4
[junit4:junit4] 2> 731916 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10041 (-1438859480804622336)]} 0 1
[junit4:junit4] 2> 731922 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10041 (-1438859480809865216)]} 0 0
[junit4:junit4] 2> 731925 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[41 (-1438859480814059520)]} 0 1
[junit4:junit4] 2> 731929 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10042 (1438859480815108096)]} 0 3
[junit4:junit4] 2> 731936 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[41 (-1438859480824545280)]} 0 1
[junit4:junit4] 2> 731938 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10042 (1438859480826642432)]} 0 1
[junit4:junit4] 2> 731940 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[42 (1438859480828739584)]} 0 1
[junit4:junit4] 2> 731946 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[42 (1438859480835031040)]} 0 1
[junit4:junit4] 2> 731955 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10042 (-1438859480845516800)]} 0 1
[junit4:junit4] 2> 731958 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10042 (-1438859480848662528)]} 0 1
[junit4:junit4] 2> 731962 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10043 (1438859480851808256)]} 0 1
[junit4:junit4] 2> 731963 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[42 (-1438859480853905408)]} 0 0
[junit4:junit4] 2> 731966 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10043 (1438859480856002560)]} 0 1
[junit4:junit4] 2> 731968 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[42 (-1438859480859148288)]} 0 0
[junit4:junit4] 2> 731972 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[43 (1438859480862294016)]} 0 1
[junit4:junit4] 2> 731976 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[43 (1438859480867536896)]} 0 1
[junit4:junit4] 2> 732021 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10044 (1438859480912625664)]} 0 2
[junit4:junit4] 2> 732025 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10044 (1438859480917868544)]} 0 1
[junit4:junit4] 2> 732035 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[44 (1438859480923111424)]} 0 6
[junit4:junit4] 2> 732042 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[44 (1438859480935694336)]} 0 2
[junit4:junit4] 2> 732094 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10045 (1438859480990220288)]} 0 2
[junit4:junit4] 2> 732105 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10045 (1438859481001754624)]} 0 2
[junit4:junit4] 2> 732111 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[45 (1438859481008046080)]} 0 2
[junit4:junit4] 2> 732118 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[45 (1438859481015386112)]} 0 1
[junit4:junit4] 2> 732137 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10045 (-1438859481036357632)]} 0 0
[junit4:junit4] 2> 732141 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10045 (-1438859481040551936)]} 0 0
[junit4:junit4] 2> 732146 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10046 (1438859481044746240)]} 0 1
[junit4:junit4] 2> 732151 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[45 (-1438859481051037696)]} 0 1
[junit4:junit4] 2> 732153 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10046 (1438859481052086272)]} 0 1
[junit4:junit4] 2> 732156 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[45 (-1438859481056280576)]} 0 0
[junit4:junit4] 2> 732161 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[46 (1438859481060474880)]} 0 1
[junit4:junit4] 2> 732165 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[46 (1438859481064669184)]} 0 1
[junit4:junit4] 2> 732221 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10047 (1438859481123389440)]} 0 2
[junit4:junit4] 2> 732225 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10047 (1438859481127583744)]} 0 1
[junit4:junit4] 2> 732233 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[47 (1438859481135972352)]} 0 2
[junit4:junit4] 2> 732242 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[47 (1438859481145409536)]} 0 1
[junit4:junit4] 2> 732264 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10046 (-1438859481168478208)]} 0 1
[junit4:junit4] 2> 732269 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10046 (-1438859481174769664)]} 0 1
[junit4:junit4] 2> 732274 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10048 (1438859481178963968)]} 0 1
[junit4:junit4] 2> 732278 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10048 (1438859481183158272)]} 0 1
[junit4:junit4] 2> 732283 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[46 (-1438859481188401152)]} 0 1
[junit4:junit4] 2> 732286 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[46 (-1438859481192595456)]} 0 0
[junit4:junit4] 2> 732290 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[48 (1438859481195741184)]} 0 1
[junit4:junit4] 2> 732294 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[48 (1438859481199935488)]} 0 1
[junit4:junit4] 2> 732327 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10049 (1438859481231392768)]} 0 4
[junit4:junit4] 2> 732337 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[49 (1438859481243975680)]} 0 2
[junit4:junit4] 2> 732339 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10049 (1438859481247121408)]} 0 1
[junit4:junit4] 2> 732344 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[49 (1438859481252364288)]} 0 1
[junit4:junit4] 2> 732437 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10047 (-1438859481349881856)]} 0 1
[junit4:junit4] 2> 732440 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10047 (-1438859481354076160)]} 0 0
[junit4:junit4] 2> 732440 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[47 (-1438859481354076160)]} 0 0
[junit4:junit4] 2> 732445 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10050 (1438859481358270464)]} 0 1
[junit4:junit4] 2> 732447 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[47 (-1438859481361416192)]} 0 0
[junit4:junit4] 2> 732449 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10050 (1438859481363513344)]} 0 1
[junit4:junit4] 2> 732452 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[50 (1438859481365610496)]} 0 1
[junit4:junit4] 2> 732464 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[50 (1438859481377144832)]} 0 2
[junit4:junit4] 2> 732508 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10048 (-1438859481425379328)]} 0 1
[junit4:junit4] 2> 732512 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10048 (-1438859481429573632)]} 0 0
[junit4:junit4] 2> 732516 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10051 (1438859481432719360)]} 0 1
[junit4:junit4] 2> 732521 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10051 (1438859481437962240)]} 0 2
[junit4:junit4] 2> 732523 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[48 (-1438859481441107968)]} 0 0
[junit4:junit4] 2> 732527 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[48 (-1438859481444253696)]} 0 1
[junit4:junit4] 2> 732531 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[51 (1438859481448448000)]} 0 2
[junit4:junit4] 2> 732535 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[51 (1438859481452642304)]} 0 1
[junit4:junit4] 2> 732586 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10052 (1438859481505071104)]} 0 2
[junit4:junit4] 2> 732591 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10052 (1438859481511362560)]} 0 2
[junit4:junit4] 2> 732598 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[52 (1438859481518702592)]} 0 2
[junit4:junit4] 2> 732602 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[52 (1438859481522896896)]} 0 1
[junit4:junit4] 2> 732675 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10053 (1438859481598394368)]} 0 2
[junit4:junit4] 2> 732686 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10053 (1438859481609928704)]} 0 2
[junit4:junit4] 2> 732686 T1782 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[53 (1438859481610977280)]} 0 2
[junit4:junit4] 2> 732694 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[53 (1438859481619365888)]} 0 2
[junit4:junit4] 2> 732732 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10049 (-1438859481660260352)]} 0 1
[junit4:junit4] 2> 732736 T1805 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10049 (-1438859481664454656)]} 0 0
[junit4:junit4] 2> 732737 T1783 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[49 (-1438859481665503232)]} 0 1
[junit4:junit4] 2> 732741 T1806 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[49 (-1438859481669697536)]} 0 1
[junit4:junit4] 2> 732741 T1787 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10054 (1438859481668648960)]} 0 1
[junit4:junit4] 2> 732751 T1785 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[54 (1438859481677037568)]} 0 4
[junit4:junit4] 2> 732754 T1802 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10054 (1438859481682280448)]} 0 2
[junit4:junit4] 2> 732756 T1801 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[54 (1438859481685426176)]} 0 1
[junit4:junit4] 2> 732853 T1784 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10055 (1438859481786089472)]} 0 2
[junit4:junit4] 2> 732854 T1786 C1584 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[55 (1438859481786089473)]} 0 2
[junit4:junit4] 2> 732858 T1804 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10055 (1438859481791332352)]} 0 1
[junit4:junit4] 2> 732859 T1803 C1585 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[55 (1438859481792380928)]} 0 1
[junit4:junit4] 2> C1586_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49652_e_ro%2Fm, base_url=http://127.0.0.1:49652/e_ro/m}
[junit4:junit4] 2> 732871 T1830 C1586 P49652 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49649/e_ro/m/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 732871 T1830 C1586 P49652 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49652/e_ro/m START replicas=[http://127.0.0.1:49649/e_ro/m/collection1/] nUpdates=100
[junit4:junit4] 2> 732872 T1830 C1586 P49652 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 732873 T1830 C1586 P49652 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 732874 T1830 C1586 P49652 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 732874 T1830 C1586 P49652 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 732874 T1830 C1586 P49652 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/tlog/tlog.0000000000000000000 refcount=1}}
[junit4:junit4] 2> 732874 T1830 C1586 P49652 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49649/e_ro/m/collection1/. core=collection1
[junit4:junit4] 2> 732875 T1830 C1586 P49652 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 732883 T1769 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 732897 T1769 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> ASYNC NEW_CORE C1587 name=collection1 org.apache.solr.core.SolrCore@7bf91eea url=http://127.0.0.1:49649/e_ro/m/collection1 node=127.0.0.1:49649_e_ro%2Fm C1587_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49649_e_ro%2Fm, base_url=http://127.0.0.1:49649/e_ro/m, leader=true}
[junit4:junit4] 2> 732912 T1805 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 732915 T1769 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=3core=collection1
[junit4:junit4] 2> 732916 T1806 C1587 P49649 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C1588 name=collection1 org.apache.solr.core.SolrCore@1a7ce791 url=http://127.0.0.1:49645/e_ro/m/collection1 node=127.0.0.1:49645_e_ro%2Fm C1588_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49645_e_ro%2Fm, base_url=http://127.0.0.1:49645/e_ro/m, leader=true}
[junit4:junit4] 2> 732936 T1783 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10050 (-1438859481874169856)]} 0 0
[junit4:junit4] 2> 732937 T1787 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[50 (-1438859481874169857)]} 0 1
[junit4:junit4] 2> 732942 T1802 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10050 (-1438859481878364160)]} 0 2
[junit4:junit4] 2> 732943 T1802 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[50 (-1438859481880461312)]} 0 1
[junit4:junit4] 2> 732949 T1785 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10056 (1438859481883607040)]} 0 5
[junit4:junit4] 2> 732951 T1782 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[56 (1438859481888849920)]} 0 1
[junit4:junit4] 2> 732956 T1804 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10056 (1438859481891995648)]} 0 3
[junit4:junit4] 2> 732957 T1803 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[56 (1438859481895141376)]} 0 2
[junit4:junit4] 2> 732960 T1806 C1587 P49649 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b948d26 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7734f6ba),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b948d26 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7734f6ba),segFN=segments_2,generation=2}
[junit4:junit4] 2> 732960 T1806 C1587 P49649 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 733027 T1786 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[57 (1438859481968541697)]} 0 2
[junit4:junit4] 2> 733028 T1784 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10057 (1438859481968541696)]} 0 3
[junit4:junit4] 2> 733033 T1805 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[57 (1438859481973784576)]} 0 2
[junit4:junit4] 2> 733034 T1802 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10057 (1438859481974833152)]} 0 2
[junit4:junit4] 2> 733045 T1806 C1587 P49649 oass.SolrIndexSearcher.<init> Opening Searcher@107433b7 realtime
[junit4:junit4] 2> 733050 T1806 C1587 P49649 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 733051 T1806 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 135
[junit4:junit4] 2> ASYNC NEW_CORE C1589 name=collection1 org.apache.solr.core.SolrCore@53198fc9 url=http://127.0.0.1:49652/e_ro/m/collection1 node=127.0.0.1:49652_e_ro%2Fm C1589_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49652_e_ro%2Fm, base_url=http://127.0.0.1:49652/e_ro/m}
[junit4:junit4] 2> 733052 T1830 C1589 P49652 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 733053 T1830 C1589 P49652 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 733056 T1804 C1587 P49649 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 733057 T1804 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 733059 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 733059 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 733059 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 733064 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 733064 T1803 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 733066 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 5
[junit4:junit4] 2> 733066 T1792 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49652_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49652/e_ro/m"}
[junit4:junit4] 2> 733066 T1830 C1589 P49652 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index.20130625173532911
[junit4:junit4] 2> 733067 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@cb689da lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fca7751) fullCopy=false
[junit4:junit4] 2> 733072 T1805 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={file=_0.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 733079 T1802 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={file=_0.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 733082 T1827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 733082 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 733082 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 733082 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 733095 T1801 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 733099 T1804 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={file=_0_1.del&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 733104 T1803 C1587 P49649 oasc.SolrCore.execute [collection1] webapp=/e_ro/m path=/replication params={file=_0.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 733105 T1830 C1589 P49652 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 733106 T1830 C1589 P49652 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 733107 T1830 C1589 P49652 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 733107 T1830 C1589 P49652 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4:junit4] 2> 733119 T1830 C1589 P49652 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 733120 T1830 C1589 P49652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 733123 T1830 C1589 P49652 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 733124 T1830 C1589 P49652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 733124 T1830 C1589 P49652 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 733133 T1830 C1589 P49652 oass.SolrIndexSearcher.<init> Opening Searcher@5f93887 main
[junit4:junit4] 2> 733135 T1829 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f93887 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c38/12)}
[junit4:junit4] 2> 733135 T1783 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[58 (1438859482081787904)]} 0 1
[junit4:junit4] 2> 733138 T1783 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10058 (1438859482083885056)]} 0 2
[junit4:junit4] 2> 733138 T1830 C1589 P49652 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index.20130625173532911 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index.20130625173532911;done=true>>]
[junit4:junit4] 2> 733140 T1830 C1589 P49652 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index.20130625173532911
[junit4:junit4] 2> 733140 T1830 C1589 P49652 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index.20130625173532911
[junit4:junit4] 2> 733141 T1805 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[58 (1438859482088079360)]} 0 1
[junit4:junit4] 2> 733143 T1805 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10058 (1438859482090176512)]} 0 1
[junit4:junit4] 2> 733147 T1830 C1589 P49652 oasc.RecoveryStrategy.replay Replaying buffered documents. core=collection1
[junit4:junit4] 2> 733148 T1838 C1589 P49652 oasu.UpdateLog$LogReplayer.doReplay WARN Starting log replay tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/tlog/tlog.0000000000000000000 refcount=2} active=true starting pos=5606
[junit4:junit4] 2> 733153 T1838 C1589 P49652 oasu.DirectUpdateHandler2.commit start commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 733155 T1838 C1589 P49652 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d5189b8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b54966c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 733155 T1838 C1589 P49652 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 733156 T1838 C1589 P49652 oass.SolrIndexSearcher.<init> Opening Searcher@67d68f53 main
[junit4:junit4] 2> 733156 T1838 C1589 P49652 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 733157 T1829 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67d68f53 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c38/12)}
[junit4:junit4] 2> 733158 T1838 C1589 P49652 oasup.LogUpdateProcessor.finish [collection1] {} 0 10
[junit4:junit4] 2> 733159 T1838 C1589 P49652 oasu.UpdateLog$LogReplayer.run WARN Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=0 positionOfStart=5606}
[junit4:junit4] 2> 733159 T1830 C1589 P49652 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 733159 T1830 C1589 P49652 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 733160 T1830 C1589 P49652 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 733162 T1830 C1589 P49652 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 733163 T1769 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@53198fc9
[junit4:junit4] 2> 733174 T1769 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=38,cumulative_deletesById=22,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 733175 T1769 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 733176 T1769 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 733176 T1769 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 733179 T1769 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 733180 T1769 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 733180 T1769 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2;done=false>>]
[junit4:junit4] 2> 733180 T1769 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2
[junit4:junit4] 2> 733181 T1769 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index;done=false>>]
[junit4:junit4] 2> 733181 T1769 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index
[junit4:junit4] 2> 733186 T1811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 733189 T1798 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 733189 T1798 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 733191 T1791 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 733191 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 733194 T1811 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 733200 T1769 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/e_ro/m,null}
[junit4:junit4] 2> 733232 T1785 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[51 (-1438859482184548352)]} 0 1
[junit4:junit4] 2> 733233 T1782 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10051 (-1438859482185596928)]} 0 0
[junit4:junit4] 2> 733237 T1806 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[51 (-1438859482189791232)]} 0 0
[junit4:junit4] 2> 733237 T1801 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10051 (-1438859482189791233)]} 0 0
[junit4:junit4] 2> 733242 T1786 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[59 (1438859482193985536)]} 0 1
[junit4:junit4] 2> 733243 T1784 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10059 (1438859482193985537)]} 0 2
[junit4:junit4] 2> 733246 T1804 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[59 (1438859482198179840)]} 0 1
[junit4:junit4] 2> 733248 T1803 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10059 (1438859482201325568)]} 0 1
[junit4:junit4] 2> 733361 T1787 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[60 (1438859482315620352)]} 0 5
[junit4:junit4] 2> 733363 T1785 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10060 (1438859482316668928)]} 0 5
[junit4:junit4] 2> 733371 T1805 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[60 (1438859482328203264)]} 0 3
[junit4:junit4] 2> 733373 T1802 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10060 (1438859482331348992)]} 0 1
[junit4:junit4] 2> 733386 T1782 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[55 (-1438859482344980480)]} 0 1
[junit4:junit4] 2> 733387 T1782 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10055 (-1438859482347077632)]} 0 0
[junit4:junit4] 2> 733389 T1801 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[55 (-1438859482349174784)]} 0 0
[junit4:junit4] 2> 733391 T1806 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10055 (-1438859482351271936)]} 0 1
[junit4:junit4] 2> 733394 T1784 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[61 (1438859482352320512)]} 0 2
[junit4:junit4] 2> 733397 T1783 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10061 (1438859482356514816)]} 0 2
[junit4:junit4] 2> 733399 T1804 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[61 (1438859482358611968)]} 0 1
[junit4:junit4] 2> 733401 T1803 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10061 (1438859482360709120)]} 0 1
[junit4:junit4] 2> 733442 T1787 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[62 (1438859482403700736)]} 0 2
[junit4:junit4] 2> 733445 T1787 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10062 (1438859482405797888)]} 0 2
[junit4:junit4] 2> 733448 T1805 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[62 (1438859482408943616)]} 0 2
[junit4:junit4] 2> 733450 T1802 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10062 (1438859482412089344)]} 0 1
[junit4:junit4] 2> 733455 T1769 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 733461 T1769 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49652
[junit4:junit4] 2> 733462 T1769 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 733462 T1769 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 733463 T1769 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781
[junit4:junit4] 2> 733463 T1769 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/solr.xml
[junit4:junit4] 2> 733463 T1769 oasc.CoreContainer.<init> New CoreContainer 1722172963
[junit4:junit4] 2> 733465 T1769 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/'
[junit4:junit4] 2> 733465 T1769 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1372203323781/'
[junit4:junit4] 2> 733552 T1786 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[57 (-1438859482520092672)]} 0 1
[junit4:junit4] 2> 733554 T1782 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10057 (-1438859482522189824)]} 0 0
[junit4:junit4] 2> 733556 T1801 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[57 (-1438859482524286976)]} 0 1
[junit4:junit4] 2> 733558 T1804 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10057 (-1438859482526384128)]} 0 1
[junit4:junit4] 2> 733560 T1784 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[63 (1438859482527432704)]} 0 1
[junit4:junit4] 2> 733565 T1783 C1588 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10063 (1438859482531627008)]} 0 3
[junit4:junit4] 2> 733566 T1803 C1587 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[63 (1438859482533724160)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C1590 name=collection1 org.apache.solr.core.SolrCore@7bf91eea url=http://127.0.0.1:49649/e_ro/m/collection1 node=127.0.0.1:49649_e_ro%2Fm C1590_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49649_e_ro%2Fm, base_url=http://127.0.0.1:49649/e_ro/m, leader=true}
[junit4:junit4] 2> 733589 T1805 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10063 (1438859482556792832)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C1591 name=collection1 org.apache.solr.core.SolrCore@1a7ce791 url=http://127.0.0.1:49645/e_ro/m/collection1 node=127.0.0.1:49645_e_ro%2Fm C1591_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49645_e_ro%2Fm, base_url=http://127.0.0.1:49645/e_ro/m, leader=true}
[junit4:junit4] 2> 733628 T1787 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[64 (1438859482597687296)]} 0 2
[junit4:junit4] 2> 733639 T1802 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[64 (1438859482610270208)]} 0 2
[junit4:junit4] 2> 733642 T1769 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 733642 T1769 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 733643 T1769 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 733643 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 733644 T1769 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 733644 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 733644 T1769 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 733645 T1769 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 733645 T1769 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 733646 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 733669 T1785 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10064 (1438859482635436032)]} 0 8
[junit4:junit4] 2> 733670 T1786 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[65 (1438859482642776064)]} 0 1
[junit4:junit4] 2> 733675 T1801 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10064 (1438859482648018944)]} 0 2
[junit4:junit4] 2> 733684 T1769 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 733684 T1769 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49642/solr
[junit4:junit4] 2> 733685 T1769 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 733685 T1806 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[65 (1438859482658504704)]} 0 1
[junit4:junit4] 2> 733694 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733695 T1849 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@283c035 name:ZooKeeperConnection Watcher:127.0.0.1:49642 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733697 T1782 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10065 (1438859482671087616)]} 0 2
[junit4:junit4] 2> 733698 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733701 T1804 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10065 (1438859482675281920)]} 0 1
[junit4:junit4] 2> 733727 T1784 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[60 (-1438859482702544896)]} 0 1
[junit4:junit4] 2> 733727 T1783 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10060 (-1438859482703593472)]} 0 0
[junit4:junit4] 2> 733735 T1769 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 733741 T1803 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[60 (-1438859482718273536)]} 0 1
[junit4:junit4] 2> 733741 T1805 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10060 (-1438859482718273537)]} 0 1
[junit4:junit4] 2> 733755 T1787 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[66 (1438859482729807872)]} 0 3
[junit4:junit4] 2> 733763 T1786 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10066 (1438859482736099328)]} 0 6
[junit4:junit4] 2> 733765 T1802 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[66 (1438859482742390784)]} 0 1
[junit4:junit4] 2> 733775 T1801 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10066 (1438859482752876544)]} 0 1
[junit4:junit4] 2> 733785 T1782 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[61 (-1438859482764410880)]} 0 0
[junit4:junit4] 2> 733797 T1784 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10061 (-1438859482776993792)]} 0 0
[junit4:junit4] 2> 733797 T1806 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[61 (-1438859482776993792)]} 0 0
[junit4:junit4] 2> 733815 T1804 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {delete=[10061 (-1438859482795868160)]} 0 0
[junit4:junit4] 2> 733816 T1783 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[67 (1438859482795868160)]} 0 1
[junit4:junit4] 2> 733817 T1769 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733820 T1851 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b9bb066 name:ZooKeeperConnection Watcher:127.0.0.1:49642/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733823 T1787 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10067 (1438859482803208192)]} 0 1
[junit4:junit4] 2> 733823 T1803 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[67 (1438859482803208192)]} 0 1
[junit4:junit4] 2> 733824 T1769 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733835 T1805 C1590 P49649 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[10067 (1438859482814742528)]} 0 3
[junit4:junit4] 2> 733848 T1769 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 733854 T1785 C1591 P49645 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_ro/m path=/update params={wt=javabin&version=2} {add=[68 (1438859482835714048)]} 0 1
[junit4:junit4] 2> 733875 T1802
[...truncated too long message...]
2> 829383 T1769 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1722172963
[junit4:junit4] 2> 829406 T1861 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:49652_e_ro%2Fm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49652/e_ro/m"}
[junit4:junit4] 2> 829415 T1851 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 829487 T1791 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 829487 T1791 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 829487 T1791 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 830388 T1769 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 830388 T1769 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 830391 T1769 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3398866b
[junit4:junit4] 2> 830402 T1769 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=43,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=45,cumulative_deletesById=32,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 830403 T1769 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 830403 T1769 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 830404 T1769 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 830411 T1769 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 830412 T1769 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 830412 T1769 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index;done=false>>]
[junit4:junit4] 2> 830413 T1769 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2/index
[junit4:junit4] 2> 830413 T1769 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2;done=false>>]
[junit4:junit4] 2> 830414 T1769 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1372203311977/jetty2
[junit4:junit4] 2> 830415 T1861 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89928716254773258-127.0.0.1:49652_e_ro%2Fm-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 830432 T1769 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/e_ro/m,null}
[junit4:junit4] 2> 830506 T1769 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 830511 T1769 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49642 49642
[junit4:junit4] 2> 830604 T1811 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 830610 T1811 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch -Dtests.seed=10E034D761A8A91B -Dtests.slow=true -Dtests.locale=es_CL -Dtests.timezone=America/Managua -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 119s | RecoveryZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: expected:<130> but was:<26>
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([10E034D761A8A91B:9106BACF16F7C927]:0)
[junit4:junit4] > at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 830769 T1769 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 118651 T1768 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 831720 T1851 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 831720 T1851 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 831721 T1851 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=MockVariableIntBlock(baseBlockSize=52), text=PostingsFormat(name=SimpleText), _version_=PostingsFormat(name=Memory doPackFST= true), rnd_b=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Memory doPackFST= true), timestamp=MockVariableIntBlock(baseBlockSize=52), id=PostingsFormat(name=Direct), a_t=MockVariableIntBlock(baseBlockSize=52), range_facet_sl=PostingsFormat(name=Direct), range_facet_si=MockVariableIntBlock(baseBlockSize=52), other_tl1=PostingsFormat(name=Memory doPackFST= true), multiDefault=MockVariableIntBlock(baseBlockSize=52), a_si=PostingsFormat(name=Memory doPackFST= true)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=es_CL, timezone=America/Managua
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=1,free=34869696,total=170483712
[junit4:junit4] 2> NOTE: All tests run in this JVM: [DocumentBuilderTest, IndexReaderFactoryTest, TestQueryUtils, TestRandomFaceting, TestSchemaNameResource, DistributedQueryElevationComponentTest, TestSolrXMLSerializer, IndexSchemaRuntimeFieldTest, TestQueryTypes, BadCopyFieldTest, SynonymTokenizerTest, TestRangeQuery, TestUniqueKeyFieldResource, DisMaxRequestHandlerTest, StatelessScriptUpdateProcessorFactoryTest, SolrIndexConfigTest, DOMUtilTest, TestCloudManagedSchemaAddField, TestRTGBase, TestMultiCoreConfBootstrap, JSONWriterTest, SpellCheckComponentTest, TestSerializedLuceneMatchVersion, TestSolrDeletionPolicy2, ReturnFieldsTest, TestCodecSupport, TestBinaryField, LeaderElectionTest, TestClassNameShortening, TestBadConfig, DefaultValueUpdateProcessorTest, PeerSyncTest, CacheHeaderTest, TestMergePolicyConfig, SchemaVersionSpecificBehaviorTest, TestJmxIntegration, TestDynamicFieldCollectionResource, TestSchemaSimilarityResource, TestDistributedSearch, TestSuggestSpellingConverter, TestFastLRUCache, TestLuceneMatchVersion, DirectSolrConnectionTest, TestAtomicUpdateErrorCases, TestAnalyzedSuggestions, TestQuerySenderListener, TestTrie, NumericFieldsTest, ClusterStateTest, TestCopyFieldCollectionResource, TestValueSourceCache, SuggesterFSTTest, SolrPluginUtilsTest, ShardRoutingTest, SpatialFilterTest, SpellPossibilityIteratorTest, FileBasedSpellCheckerTest, TestDefaultSimilarityFactory, TestDocSet, TestRealTimeGet, MoreLikeThisHandlerTest, TestReload, WordBreakSolrSpellCheckerTest, TestFieldTypeCollectionResource, ZkNodePropsTest, UpdateParamsTest, TestExtendedDismaxParser, TestWordDelimiterFilterFactory, DistributedTermsComponentTest, DirectUpdateHandlerOptimizeTest, BasicDistributedZk2Test, OverseerCollectionProcessorTest, SearchHandlerTest, TermVectorComponentTest, SliceStateUpdateTest, SOLR749Test, RecoveryZkTest]
[junit4:junit4] Completed in 119.72s, 1 test, 1 failure <<< FAILURES!
[...truncated 681 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 87 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure