You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/24 07:22:52 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 6211
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6211/
Java: 32bit/jdk1.6.0_45 -client -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:44934/h_/collection1lastClient and got 256 from http://127.0.0.1:47592/h_/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:44934/h_/collection1lastClient and got 256 from http://127.0.0.1:47592/h_/collection1
at __randomizedtesting.SeedInfo.seed([D3EE7114E39CA0F3:5208FF0C94C3C0CF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 8918 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 250825 T1482 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /h_/
[junit4:junit4] 2> 250828 T1482 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372049938354
[junit4:junit4] 2> 250829 T1482 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 250829 T1483 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 250930 T1482 oasc.ZkTestServer.run start zk server on port:45612
[junit4:junit4] 2> 250931 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 251067 T1489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@542c8d name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 251068 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 251068 T1482 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 251072 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 251073 T1491 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@125c485 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 251073 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 251074 T1482 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 251077 T1482 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 251080 T1482 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 251081 T1482 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 251083 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 251083 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 251086 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 251087 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 251089 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 251089 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 251091 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 251091 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 251093 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 251093 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 251095 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 251095 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 251097 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 251098 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 251100 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 251100 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 251102 T1482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 251103 T1482 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 251182 T1482 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 251185 T1482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35478
[junit4:junit4] 2> 251185 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 251186 T1482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 251186 T1482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630
[junit4:junit4] 2> 251186 T1482 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/solr.xml
[junit4:junit4] 2> 251187 T1482 oasc.CoreContainer.<init> New CoreContainer 27529196
[junit4:junit4] 2> 251187 T1482 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/'
[junit4:junit4] 2> 251188 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/'
[junit4:junit4] 2> 251250 T1482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 251251 T1482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 251251 T1482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 251252 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 251252 T1482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 251253 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 251253 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 251262 T1482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 251262 T1482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 251263 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 251266 T1482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 251267 T1482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45612/solr
[junit4:junit4] 2> 251267 T1482 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 251268 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 251270 T1502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c408fb name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 251270 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 251271 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 251274 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 251275 T1504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75e921 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 251276 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 251277 T1482 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 251279 T1482 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 251281 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 251283 T1482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35478_h_
[junit4:junit4] 2> 251283 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35478_h_
[junit4:junit4] 2> 251285 T1482 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 251289 T1482 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 251291 T1482 oasc.Overseer.start Overseer (id=89918664760098819-127.0.0.1:35478_h_-n_0000000000) starting
[junit4:junit4] 2> 251293 T1482 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 251295 T1506 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 251296 T1482 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 251297 T1482 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 251298 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 251300 T1505 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 251303 T1507 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 251303 T1507 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 252803 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 252803 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35478_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35478/h_"}
[junit4:junit4] 2> 252804 T1505 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 252804 T1505 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 252807 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 253304 T1507 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/collection1
[junit4:junit4] 2> 253304 T1507 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 253305 T1507 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 253305 T1507 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 253306 T1507 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/collection1/'
[junit4:junit4] 2> 253307 T1507 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/collection1/lib/README' to classloader
[junit4:junit4] 2> 253307 T1507 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 253346 T1507 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 253396 T1507 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 253398 T1507 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 253402 T1507 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 253910 T1507 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 253911 T1507 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 253911 T1507 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 253918 T1507 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 253921 T1507 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 253931 T1507 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 253934 T1507 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 253937 T1507 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 253938 T1507 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 253939 T1507 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 253939 T1507 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 253940 T1507 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 253940 T1507 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 253940 T1507 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 253947 T1507 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372049938630/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/
[junit4:junit4] 2> 253947 T1507 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f2225f
[junit4:junit4] 2> 253947 T1507 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 253948 T1507 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data
[junit4:junit4] 2> 253948 T1507 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index/
[junit4:junit4] 2> 253948 T1507 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 253949 T1507 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index
[junit4:junit4] 2> 253956 T1507 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 253957 T1507 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 253959 T1507 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 253959 T1507 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 253960 T1507 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 253961 T1507 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 253961 T1507 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 253962 T1507 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 253962 T1507 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 253963 T1507 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 253963 T1507 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 253966 T1507 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 253970 T1507 oass.SolrIndexSearcher.<init> Opening Searcher@486937 main
[junit4:junit4] 2> 253971 T1507 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 253971 T1507 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 253975 T1508 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@486937 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 253977 T1507 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 253977 T1507 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35478/h_ collection:control_collection shard:shard1
[junit4:junit4] 2> 253978 T1507 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 253984 T1507 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 253986 T1507 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 253986 T1507 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 253986 T1507 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35478/h_/collection1/
[junit4:junit4] 2> 253986 T1507 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 253987 T1507 oasc.SyncStrategy.syncToMe http://127.0.0.1:35478/h_/collection1/ has no replicas
[junit4:junit4] 2> 253987 T1507 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35478/h_/collection1/
[junit4:junit4] 2> 253987 T1507 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 254310 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 254316 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 254342 T1507 oasc.ZkController.register We are http://127.0.0.1:35478/h_/collection1/ and leader is http://127.0.0.1:35478/h_/collection1/
[junit4:junit4] 2> 254342 T1507 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35478/h_
[junit4:junit4] 2> 254342 T1507 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 254342 T1507 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 254342 T1507 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 254344 T1507 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 254345 T1482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 254345 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 254346 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 254349 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 254350 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 254351 T1511 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198cf95 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 254351 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 254352 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 254354 T1482 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 254423 T1482 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 254425 T1482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44934
[junit4:junit4] 2> 254426 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 254426 T1482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 254426 T1482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879
[junit4:junit4] 2> 254427 T1482 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/solr.xml
[junit4:junit4] 2> 254427 T1482 oasc.CoreContainer.<init> New CoreContainer 29758294
[junit4:junit4] 2> 254428 T1482 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/'
[junit4:junit4] 2> 254428 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/'
[junit4:junit4] 2> 254496 T1482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 254496 T1482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 254497 T1482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 254497 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 254497 T1482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 254498 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 254498 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 254499 T1482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 254499 T1482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 254499 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 254502 T1482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 254503 T1482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45612/solr
[junit4:junit4] 2> 254503 T1482 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 254504 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 254505 T1522 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1df0412 name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 254506 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 254507 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 254510 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 254511 T1524 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@221801 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 254511 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 254514 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 255516 T1482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44934_h_
[junit4:junit4] 2> 255517 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44934_h_
[junit4:junit4] 2> 255520 T1511 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 255520 T1504 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 255520 T1524 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 255521 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 255531 T1525 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 255532 T1525 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 255819 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 255820 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35478_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35478/h_"}
[junit4:junit4] 2> 255823 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44934_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44934/h_"}
[junit4:junit4] 2> 255823 T1505 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 255823 T1505 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 255827 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 255827 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 255827 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 256533 T1525 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/collection1
[junit4:junit4] 2> 256533 T1525 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 256534 T1525 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 256534 T1525 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 256535 T1525 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/collection1/'
[junit4:junit4] 2> 256536 T1525 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/collection1/lib/README' to classloader
[junit4:junit4] 2> 256536 T1525 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 256576 T1525 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 256621 T1525 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 256622 T1525 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 256627 T1525 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 257136 T1525 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 257137 T1525 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 257137 T1525 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 257144 T1525 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 257147 T1525 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 257157 T1525 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 257160 T1525 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 257163 T1525 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 257164 T1525 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 257164 T1525 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 257164 T1525 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 257165 T1525 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 257165 T1525 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 257166 T1525 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 257166 T1525 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372049941879/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/
[junit4:junit4] 2> 257166 T1525 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f2225f
[junit4:junit4] 2> 257167 T1525 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 257167 T1525 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1
[junit4:junit4] 2> 257167 T1525 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index/
[junit4:junit4] 2> 257168 T1525 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 257168 T1525 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index
[junit4:junit4] 2> 257171 T1525 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 257171 T1525 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 257173 T1525 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 257173 T1525 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 257174 T1525 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 257175 T1525 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 257176 T1525 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 257176 T1525 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 257176 T1525 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 257177 T1525 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 257177 T1525 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 257180 T1525 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 257183 T1525 oass.SolrIndexSearcher.<init> Opening Searcher@31d6ec main
[junit4:junit4] 2> 257184 T1525 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 257184 T1525 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 257194 T1526 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31d6ec main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 257196 T1525 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 257196 T1525 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44934/h_ collection:collection1 shard:shard1
[junit4:junit4] 2> 257197 T1525 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 257201 T1525 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 257203 T1525 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 257203 T1525 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 257203 T1525 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 257204 T1525 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 257204 T1525 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/ has no replicas
[junit4:junit4] 2> 257204 T1525 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 257204 T1525 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 257330 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 257335 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 257336 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 257335 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 257362 T1525 oasc.ZkController.register We are http://127.0.0.1:44934/h_/collection1/ and leader is http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 257362 T1525 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44934/h_
[junit4:junit4] 2> 257362 T1525 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 257362 T1525 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 257362 T1525 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 257364 T1525 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 257365 T1482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 257366 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 257366 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 257440 T1482 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 257442 T1482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55964
[junit4:junit4] 2> 257443 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 257443 T1482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 257444 T1482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894
[junit4:junit4] 2> 257444 T1482 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/solr.xml
[junit4:junit4] 2> 257444 T1482 oasc.CoreContainer.<init> New CoreContainer 1736245
[junit4:junit4] 2> 257445 T1482 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/'
[junit4:junit4] 2> 257445 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/'
[junit4:junit4] 2> 257506 T1482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 257506 T1482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 257507 T1482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 257507 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 257507 T1482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 257508 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 257508 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 257508 T1482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 257509 T1482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 257509 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 257512 T1482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 257513 T1482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45612/solr
[junit4:junit4] 2> 257513 T1482 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 257514 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 257515 T1538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@167f54a name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 257515 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 257517 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 257520 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 257521 T1540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c767c9 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 257521 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 257524 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 258526 T1482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55964_h_
[junit4:junit4] 2> 258528 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55964_h_
[junit4:junit4] 2> 258529 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 258529 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 258530 T1504 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 258531 T1540 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 258531 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 258531 T1511 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 258531 T1524 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 258536 T1541 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 258536 T1541 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 258839 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 258840 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44934_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44934/h_"}
[junit4:junit4] 2> 258843 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55964_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55964/h_"}
[junit4:junit4] 2> 258843 T1505 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 258843 T1505 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 258845 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 258846 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 258845 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 258845 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 259537 T1541 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/collection1
[junit4:junit4] 2> 259537 T1541 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 259538 T1541 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 259538 T1541 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 259539 T1541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/collection1/'
[junit4:junit4] 2> 259540 T1541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/collection1/lib/README' to classloader
[junit4:junit4] 2> 259540 T1541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 259580 T1541 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 259630 T1541 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 259631 T1541 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 259636 T1541 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 260154 T1541 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 260154 T1541 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 260155 T1541 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 260162 T1541 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 260165 T1541 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 260174 T1541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 260177 T1541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 260180 T1541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 260181 T1541 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 260181 T1541 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 260182 T1541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 260182 T1541 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 260183 T1541 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 260183 T1541 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 260183 T1541 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372049944894/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/
[junit4:junit4] 2> 260183 T1541 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f2225f
[junit4:junit4] 2> 260184 T1541 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 260184 T1541 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2
[junit4:junit4] 2> 260185 T1541 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index/
[junit4:junit4] 2> 260185 T1541 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 260186 T1541 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index
[junit4:junit4] 2> 260189 T1541 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 260190 T1541 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 260191 T1541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 260192 T1541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 260192 T1541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 260193 T1541 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 260193 T1541 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 260194 T1541 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 260194 T1541 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 260194 T1541 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 260195 T1541 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 260197 T1541 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 260200 T1541 oass.SolrIndexSearcher.<init> Opening Searcher@19758e4 main
[junit4:junit4] 2> 260200 T1541 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 260200 T1541 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 260210 T1542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19758e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 260212 T1541 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 260212 T1541 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55964/h_ collection:collection1 shard:shard1
[junit4:junit4] 2> 260214 T1541 oasc.ZkController.register We are http://127.0.0.1:55964/h_/collection1/ and leader is http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 260214 T1541 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55964/h_
[junit4:junit4] 2> 260215 T1541 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 260215 T1541 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4525 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4525_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 260215 T1543 C4525 P55964 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 260216 T1543 C4525 P55964 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 260216 T1541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 260216 T1543 C4525 P55964 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 260217 T1543 C4525 P55964 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 260217 T1482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 260217 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 260218 T1543 C4525 P55964 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 260218 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 260222 T1518 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 260291 T1482 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 260293 T1482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47592
[junit4:junit4] 2> 260294 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 260294 T1482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 260294 T1482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746
[junit4:junit4] 2> 260295 T1482 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/solr.xml
[junit4:junit4] 2> 260295 T1482 oasc.CoreContainer.<init> New CoreContainer 26683896
[junit4:junit4] 2> 260296 T1482 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/'
[junit4:junit4] 2> 260296 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/'
[junit4:junit4] 2> 260354 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 260355 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55964_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55964/h_"}
[junit4:junit4] 2> 260357 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 260357 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 260357 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 260357 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 260361 T1482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 260362 T1482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 260362 T1482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 260363 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 260363 T1482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 260363 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 260364 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 260364 T1482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 260365 T1482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 260365 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 260368 T1482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 260369 T1482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45612/solr
[junit4:junit4] 2> 260369 T1482 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 260370 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 260371 T1555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b63799 name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 260371 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 260373 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 260376 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 260377 T1557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a3e9d6 name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 260377 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 260380 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 261222 T1518 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 261222 T1518 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:55964_h_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 261382 T1482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47592_h_
[junit4:junit4] 2> 261383 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47592_h_
[junit4:junit4] 2> 261385 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 261385 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 261385 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 261386 T1557 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 261386 T1504 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 261387 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 261387 T1511 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 261388 T1540 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 261388 T1524 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 261391 T1558 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 261391 T1558 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 261861 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 261862 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47592_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47592/h_"}
[junit4:junit4] 2> 261862 T1505 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 261862 T1505 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 261865 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 261865 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 261865 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 261865 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 261866 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 262392 T1558 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/collection1
[junit4:junit4] 2> 262393 T1558 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 262393 T1558 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 262393 T1558 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 262394 T1558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/collection1/'
[junit4:junit4] 2> 262395 T1558 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/collection1/lib/README' to classloader
[junit4:junit4] 2> 262395 T1558 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 262438 T1558 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 262487 T1558 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 262488 T1558 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 262493 T1558 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 263052 T1558 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 263053 T1558 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 263054 T1558 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 263066 T1558 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 263070 T1558 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 263080 T1558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 263083 T1558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 263086 T1558 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 263087 T1558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 263087 T1558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 263087 T1558 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 263088 T1558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 263088 T1558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 263089 T1558 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 263089 T1558 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372049947746/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/
[junit4:junit4] 2> 263089 T1558 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f2225f
[junit4:junit4] 2> 263090 T1558 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 263090 T1558 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3
[junit4:junit4] 2> 263090 T1558 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index/
[junit4:junit4] 2> 263091 T1558 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 263091 T1558 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index
[junit4:junit4] 2> 263094 T1558 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 263094 T1558 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 263096 T1558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 263096 T1558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 263096 T1558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 263097 T1558 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 263098 T1558 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 263098 T1558 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 263098 T1558 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 263099 T1558 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 263099 T1558 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 263101 T1558 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 263104 T1558 oass.SolrIndexSearcher.<init> Opening Searcher@1e233c main
[junit4:junit4] 2> 263104 T1558 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 263105 T1558 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 263111 T1559 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e233c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 263113 T1558 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 263113 T1558 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47592/h_ collection:collection1 shard:shard1
[junit4:junit4] 2> 263116 T1558 oasc.ZkController.register We are http://127.0.0.1:47592/h_/collection1/ and leader is http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 263116 T1558 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47592/h_
[junit4:junit4] 2> 263116 T1558 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 263116 T1558 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4526 name=collection1 org.apache.solr.core.SolrCore@1f1625 url=http://127.0.0.1:47592/h_/collection1 node=127.0.0.1:47592_h_ C4526_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:47592_h_, base_url=http://127.0.0.1:47592/h_}
[junit4:junit4] 2> 263117 T1560 C4526 P47592 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 263117 T1560 C4526 P47592 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 263117 T1558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 263118 T1560 C4526 P47592 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 263118 T1560 C4526 P47592 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 263119 T1482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 263119 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 263119 T1560 C4526 P47592 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 263119 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 263128 T1517 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 263204 T1482 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 263206 T1482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43552
[junit4:junit4] 2> 263207 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 263207 T1482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 263207 T1482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647
[junit4:junit4] 2> 263208 T1482 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/solr.xml
[junit4:junit4] 2> 263208 T1482 oasc.CoreContainer.<init> New CoreContainer 21186305
[junit4:junit4] 2> 263209 T1482 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/'
[junit4:junit4] 2> 263209 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/'
[junit4:junit4] 2> ASYNC NEW_CORE C4527 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4527_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 263223 T1543 C4527 P55964 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44934/h_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 263224 T1543 C4527 P55964 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55964/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 263224 T1543 C4527 P55964 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 263224 T1543 C4527 P55964 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 263225 T1543 C4527 P55964 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 263225 T1543 C4527 P55964 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 263225 T1543 C4527 P55964 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 263226 T1543 C4527 P55964 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44934/h_/collection1/. core=collection1
[junit4:junit4] 2> 263226 T1543 C4527 P55964 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4528 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4528_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 263230 T1519 C4528 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 263231 T1518 C4528 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4529 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4529_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 263238 T1518 C4529 P44934 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 263238 T1518 C4529 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 263246 T1518 C4529 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 263247 T1518 C4529 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 263248 T1518 C4529 P44934 oass.SolrIndexSearcher.<init> Opening Searcher@fa24f9 realtime
[junit4:junit4] 2> 263248 T1518 C4529 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 263248 T1518 C4529 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> ASYNC NEW_CORE C4530 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4530_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 263249 T1543 C4530 P55964 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 263249 T1543 C4530 P55964 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 263250 T1518 C4529 P44934 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 263251 T1518 C4529 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 263251 T1543 C4530 P55964 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 263251 T1543 C4530 P55964 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 263251 T1543 C4530 P55964 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 263253 T1519 C4529 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 263254 T1543 C4530 P55964 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 263254 T1543 C4530 P55964 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779
[junit4:junit4] 2> 263255 T1543 C4530 P55964 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779 lockFactory=org.apache.lucene.store.NativeFSLockFactory@120aa46 fullCopy=false
[junit4:junit4] 2> 263257 T1520 C4529 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C4531 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4531_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 263258 T1543 C4531 P55964 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 263267 T1543 C4531 P55964 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 263268 T1543 C4531 P55964 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 263269 T1543 C4531 P55964 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 263269 T1543 C4531 P55964 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 263270 T1543 C4531 P55964 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 263270 T1543 C4531 P55964 oass.SolrIndexSearcher.<init> Opening Searcher@1d678b6 main
[junit4:junit4] 2> 263272 T1542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d678b6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 263272 T1543 C4531 P55964 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779;done=true>>]
[junit4:junit4] 2> 263273 T1543 C4531 P55964 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779
[junit4:junit4] 2> 263273 T1543 C4531 P55964 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index.20130624145910779
[junit4:junit4] 2> 263273 T1543 C4531 P55964 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 263273 T1543 C4531 P55964 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 263274 T1543 C4531 P55964 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 263274 T1543 C4531 P55964 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 263276 T1543 C4531 P55964 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 263282 T1482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 263282 T1482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 263283 T1482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 263283 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 263283 T1482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 263284 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 263284 T1482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 263285 T1482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 263285 T1482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 263285 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 263288 T1482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 263289 T1482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45612/solr
[junit4:junit4] 2> 263289 T1482 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 263290 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 263292 T1574 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e4e741 name:ZooKeeperConnection Watcher:127.0.0.1:45612 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 263292 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 263293 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 263296 T1482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 263297 T1576 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12b93cc name:ZooKeeperConnection Watcher:127.0.0.1:45612/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 263297 T1482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 263300 T1482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 263369 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 263370 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47592_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47592/h_"}
[junit4:junit4] 2> 263372 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55964_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55964/h_"}
[junit4:junit4] 2> 263387 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 263387 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 263387 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 263387 T1576 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 263387 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 263387 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 264128 T1517 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 264129 T1517 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:47592_h_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 264302 T1482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43552_h_
[junit4:junit4] 2> 264304 T1482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43552_h_
[junit4:junit4] 2> 264306 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 264306 T1576 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 264306 T1504 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264306 T1557 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264306 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 264306 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 264307 T1511 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264307 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264306 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264308 T1576 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264309 T1540 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264309 T1524 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 264312 T1577 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 264312 T1577 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 264901 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 264902 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43552_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43552/h_"}
[junit4:junit4] 2> 264902 T1505 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 264902 T1505 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 264905 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264905 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264905 T1576 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264905 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264905 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 264905 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 265313 T1577 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/collection1
[junit4:junit4] 2> 265314 T1577 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 265315 T1577 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 265315 T1577 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 265316 T1577 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/collection1/'
[junit4:junit4] 2> 265317 T1577 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/collection1/lib/README' to classloader
[junit4:junit4] 2> 265318 T1577 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 265362 T1577 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 265416 T1577 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 265418 T1577 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 265422 T1577 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 265967 T1577 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 265967 T1577 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 265968 T1577 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 265974 T1577 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 265977 T1577 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 265987 T1577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 265990 T1577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 265993 T1577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 265993 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 265994 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 265994 T1577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 265995 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 265995 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 265995 T1577 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 265995 T1577 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372049950647/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/
[junit4:junit4] 2> 265996 T1577 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f2225f
[junit4:junit4] 2> 265996 T1577 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 265996 T1577 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4
[junit4:junit4] 2> 265997 T1577 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index/
[junit4:junit4] 2> 265997 T1577 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 265997 T1577 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index
[junit4:junit4] 2> 266001 T1577 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 266001 T1577 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 266003 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 266003 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 266004 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 266004 T1577 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 266005 T1577 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 266005 T1577 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 266005 T1577 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 266006 T1577 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 266006 T1577 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 266015 T1577 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 266018 T1577 oass.SolrIndexSearcher.<init> Opening Searcher@c9a1b0 main
[junit4:junit4] 2> 266019 T1577 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 266019 T1577 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 266023 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c9a1b0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 266026 T1577 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 266026 T1577 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43552/h_ collection:collection1 shard:shard1
[junit4:junit4] 2> 266028 T1577 oasc.ZkController.register We are http://127.0.0.1:43552/h_/collection1/ and leader is http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 266028 T1577 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43552/h_
[junit4:junit4] 2> 266028 T1577 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 266028 T1577 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4532 name=collection1 org.apache.solr.core.SolrCore@14f66aa url=http://127.0.0.1:43552/h_/collection1 node=127.0.0.1:43552_h_ C4532_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 266029 T1579 C4532 P43552 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 266029 T1577 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 266029 T1579 C4532 P43552 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 266030 T1579 C4532 P43552 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 266030 T1579 C4532 P43552 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 266030 T1482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 266031 T1482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 266031 T1579 C4532 P43552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 266031 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 266034 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 266035 T1520 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 266035 T1482 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 266036 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 266036 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C4533 name=collection1 org.apache.solr.core.SolrCore@1f1625 url=http://127.0.0.1:47592/h_/collection1 node=127.0.0.1:47592_h_ C4533_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:47592_h_, base_url=http://127.0.0.1:47592/h_}
[junit4:junit4] 2> 266130 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44934/h_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 266130 T1560 C4533 P47592 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47592/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 266130 T1560 C4533 P47592 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 266131 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 266131 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 266131 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 266131 T1560 C4533 P47592 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 266132 T1560 C4533 P47592 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44934/h_/collection1/. core=collection1
[junit4:junit4] 2> 266132 T1560 C4533 P47592 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4534 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4534_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 266133 T1518 C4534 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 266135 T1519 C4534 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 266146 T1519 C4534 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 266147 T1519 C4534 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 266147 T1519 C4534 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 266148 T1519 C4534 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 266148 T1560 C4533 P47592 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 266149 T1560 C4533 P47592 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 266150 T1518 C4534 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 266151 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 266151 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 266151 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 266153 T1519 C4534 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=0
[junit4:junit4] 2> 266154 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 266155 T1560 C4533 P47592 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679
[junit4:junit4] 2> 266155 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a0c01a fullCopy=false
[junit4:junit4] 2> 266157 T1517 C4534 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 266158 T1560 C4533 P47592 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 266160 T1560 C4533 P47592 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 266160 T1560 C4533 P47592 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 266162 T1560 C4533 P47592 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 266162 T1560 C4533 P47592 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 266163 T1560 C4533 P47592 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 266163 T1560 C4533 P47592 oass.SolrIndexSearcher.<init> Opening Searcher@5ecff1 main
[junit4:junit4] 2> 266164 T1559 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ecff1 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 266164 T1560 C4533 P47592 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679;done=true>>]
[junit4:junit4] 2> 266165 T1560 C4533 P47592 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679
[junit4:junit4] 2> 266165 T1560 C4533 P47592 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index.20130624145913679
[junit4:junit4] 2> 266165 T1560 C4533 P47592 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 266166 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 266166 T1560 C4533 P47592 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 266166 T1560 C4533 P47592 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 266167 T1560 C4533 P47592 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 266409 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 266409 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43552_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43552/h_"}
[junit4:junit4] 2> 266412 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47592_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47592/h_"}
[junit4:junit4] 2> 266414 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 266414 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 266414 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 266414 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 266414 T1576 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 266414 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 267035 T1520 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 267036 T1520 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:43552_h_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 267038 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 268039 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C4532_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 269037 T1579 C4532 P43552 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44934/h_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 269037 T1579 C4532 P43552 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43552/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 269038 T1579 C4532 P43552 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 269038 T1579 C4532 P43552 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 269038 T1579 C4532 P43552 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 269039 T1579 C4532 P43552 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 269039 T1579 C4532 P43552 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 269039 T1579 C4532 P43552 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44934/h_/collection1/. core=collection1
[junit4:junit4] 2> 269039 T1579 C4532 P43552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 269039 T1517 C4534 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 269041 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 269045 T1518 C4534 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 269076 T1518 C4534 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> ASYNC NEW_CORE C4535 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4535_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 269085 T1518 C4535 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 269085 T1518 C4535 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 269086 T1518 C4535 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 41
[junit4:junit4] 2> ASYNC NEW_CORE C4536 name=collection1 org.apache.solr.core.SolrCore@14f66aa url=http://127.0.0.1:43552/h_/collection1 node=127.0.0.1:43552_h_ C4536_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 269086 T1579 C4536 P43552 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 269087 T1579 C4536 P43552 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 269089 T1517 C4535 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 269089 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 269089 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 269090 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 269092 T1519 C4535 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 269092 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 269093 T1579 C4536 P43552 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617
[junit4:junit4] 2> 269093 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d296e fullCopy=false
[junit4:junit4] 2> 269096 T1518 C4535 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 269097 T1579 C4536 P43552 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 269099 T1579 C4536 P43552 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 269099 T1579 C4536 P43552 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 269101 T1579 C4536 P43552 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 269102 T1579 C4536 P43552 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 269102 T1579 C4536 P43552 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 269103 T1579 C4536 P43552 oass.SolrIndexSearcher.<init> Opening Searcher@119cdaf main
[junit4:junit4] 2> 269104 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@119cdaf main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 269105 T1579 C4536 P43552 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617;done=true>>]
[junit4:junit4] 2> 269105 T1579 C4536 P43552 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617
[junit4:junit4] 2> 269105 T1579 C4536 P43552 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index.20130624145916617
[junit4:junit4] 2> 269106 T1579 C4536 P43552 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 269106 T1579 C4536 P43552 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 269106 T1579 C4536 P43552 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 269107 T1579 C4536 P43552 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 269108 T1579 C4536 P43552 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 269420 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 269421 T1505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43552_h_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43552/h_"}
[junit4:junit4] 2> 269440 T1511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 269440 T1540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 269440 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 269440 T1576 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 269440 T1524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 269440 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 270042 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 270043 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C4537 name=collection1 org.apache.solr.core.SolrCore@13b502f url=http://127.0.0.1:35478/h_/collection1 node=127.0.0.1:35478_h_ C4537_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:35478_h_, base_url=http://127.0.0.1:35478/h_, leader=true}
[junit4:junit4] 2> 270050 T1498 C4537 P35478 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 270052 T1498 C4537 P35478 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 270052 T1498 C4537 P35478 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 270056 T1498 C4537 P35478 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 270057 T1498 C4537 P35478 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 270058 T1498 C4537 P35478 oass.SolrIndexSearcher.<init> Opening Searcher@178587 main
[junit4:junit4] 2> 270058 T1498 C4537 P35478 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 270059 T1508 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@178587 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 270059 T1498 C4537 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 270065 T1520 C4535 P44934 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44934/h_/collection1/, StdNode: http://127.0.0.1:55964/h_/collection1/, StdNode: http://127.0.0.1:47592/h_/collection1/, StdNode: http://127.0.0.1:43552/h_/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 270069 T1517 C4535 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4538 name=collection1 org.apache.solr.core.SolrCore@14f66aa url=http://127.0.0.1:43552/h_/collection1 node=127.0.0.1:43552_h_ C4538_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 270079 T1570 C4538 P43552 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4539 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4539_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 270079 T1533 C4539 P55964 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4540 name=collection1 org.apache.solr.core.SolrCore@1f1625 url=http://127.0.0.1:47592/h_/collection1 node=127.0.0.1:47592_h_ C4540_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47592_h_, base_url=http://127.0.0.1:47592/h_}
[junit4:junit4] 2> 270081 T1552 C4540 P47592 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4541 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4541_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 270086 T1517 C4541 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 270087 T1517 C4541 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 270088 T1517 C4541 P44934 oass.SolrIndexSearcher.<init> Opening Searcher@3611df main
[junit4:junit4] 2> 270088 T1517 C4541 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 270089 T1526 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3611df main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 270089 T1517 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 270107 T1570 C4538 P43552 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 270107 T1570 C4538 P43552 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 270108 T1570 C4538 P43552 oass.SolrIndexSearcher.<init> Opening Searcher@1a7ce95 main
[junit4:junit4] 2> 270108 T1570 C4538 P43552 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 270109 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a7ce95 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 270109 T1570 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 30
[junit4:junit4] 2> 270116 T1552 C4540 P47592 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 270116 T1533 C4539 P55964 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 270117 T1552 C4540 P47592 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 270117 T1533 C4539 P55964 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 270118 T1552 C4540 P47592 oass.SolrIndexSearcher.<init> Opening Searcher@164035a main
[junit4:junit4] 2> 270118 T1533 C4539 P55964 oass.SolrIndexSearcher.<init> Opening Searcher@12bce71 main
[junit4:junit4] 2> 270118 T1552 C4540 P47592 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 270119 T1559 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@164035a main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 270119 T1533 C4539 P55964 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 270120 T1552 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4] 2> 270120 T1542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12bce71 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 270121 T1533 C4539 P55964 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 42
[junit4:junit4] 2> 270121 T1520 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 56
[junit4:junit4] 2> 270122 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 270124 T1519 C4541 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 270125 T1535 C4539 P55964 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 270127 T1553 C4540 P47592 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 270128 T1568 C4538 P43552 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C4542 name=collection1 org.apache.solr.core.SolrCore@13b502f url=http://127.0.0.1:35478/h_/collection1 node=127.0.0.1:35478_h_ C4542_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:35478_h_, base_url=http://127.0.0.1:35478/h_, leader=true}
[junit4:junit4] 2> 272132 T1499 C4542 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438698658496249856)} 0 2
[junit4:junit4] 2> 272139 T1535 C4539 P55964 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={update.distrib=FROMLEADER&_version_=-1438698658501492736&update.from=http://127.0.0.1:44934/h_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438698658501492736)} 0 0
[junit4:junit4] 2> 272140 T1569 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={update.distrib=FROMLEADER&_version_=-1438698658501492736&update.from=http://127.0.0.1:44934/h_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438698658501492736)} 0 1
[junit4:junit4] 2> 272139 T1552 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={update.distrib=FROMLEADER&_version_=-1438698658501492736&update.from=http://127.0.0.1:44934/h_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438698658501492736)} 0 0
[junit4:junit4] 2> 272141 T1519 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438698658501492736)} 0 5
[junit4:junit4] 2> 272141 T1568 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 7
[junit4:junit4] 2> 272145 T1498 C4542 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[0 (1438698658508832768)]} 0 2
[junit4:junit4] 2> 272155 T1570 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438698658517221376)]} 0 1
[junit4:junit4] 2> 272156 T1553 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438698658517221376)]} 0 2
[junit4:junit4] 2> 272155 T1536 C4539 P55964 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438698658517221376)]} 0 1
[junit4:junit4] 2> 272157 T1518 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:43552/h_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438698658517221376)]} 0 6
[junit4:junit4] 2> 272157 T1567 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[0]} 0 10
[junit4:junit4] 2> 272160 T1500 C4542 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[1 (1438698658525609984)]} 0 1
[junit4:junit4] 2> 272164 T1550 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438698658528755712)]} 0 0
[junit4:junit4] 2> 272165 T1569 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438698658528755712)]} 0 1
[junit4:junit4] 2> 272165 T1533 C4539 P55964 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438698658528755712)]} 0 1
[junit4:junit4] 2> 272166 T1517 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[1 (1438698658528755712)]} 0 4
[junit4:junit4] 2> 272168 T1497 C4542 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[2 (1438698658535047168)]} 0 1
[junit4:junit4] 2> 272171 T1520 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:55964/h_/collection1/]
[junit4:junit4] 2> 272171 T1520 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:55964/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:true
[junit4:junit4] 2> 272172 T1520 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:47592/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:false
[junit4:junit4] 2> 272172 T1520 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:43552/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:false
[junit4:junit4] 2> 272175 T1568 C4538 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[2 (1438698658539241472)]} 0 1
[junit4:junit4] 2> 272175 T1552 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[2 (1438698658539241472)]} 0 1
[junit4:junit4] 2> 272176 T1520 C4541 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:47592/h_/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[2 (1438698658539241472)]} 0 5
[junit4:junit4] 2> 272176 T1551 C4540 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[2]} 0 7
[junit4:junit4] 2> 272179 T1499 C4542 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={wt=javabin&version=2} {add=[3 (1438698658545532928)]} 0 1
[junit4:junit4] 2> 272182 T1519 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:55964/h_/collection1/]
[junit4:junit4] 2> 272183 T1519 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:55964/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:true
[junit4:junit4] 2> 272183 T1519 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:47592/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:false
[junit4:junit4] 2> 272183 T1519 C4541 P44934 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:43552/h_/collection1/ against:[http://127.0.0.1:55964/h_/collection1/] result:false
[junit4:junit4] 2> ASYNC NEW_CORE C4543 name=collection1 org.apache.solr.core.SolrCore@14f66aa url=http://127.0.0.1:43552/h_/collection1 node=127.0.0.1:43552_h_ C4543_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 272194 T1570 C4543 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[3 (1438698658551824384)]} 0 8
[junit4:junit4] 2> ASYNC NEW_CORE C4544 name=collection1 org.apache.solr.core.SolrCore@1f1625 url=http://127.0.0.1:47592/h_/collection1 node=127.0.0.1:47592_h_ C4544_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47592_h_, base_url=http://127.0.0.1:47592/h_}
[junit4:junit4] 2> 272194 T1550 C4544 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:44934/h_/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[3 (1438698658551824384)]} 0 8
[junit4:junit4] 2> ASYNC NEW_CORE C4545 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4545_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 272195 T1519 C4545 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={distrib.from=http://127.0.0.1:47592/h_/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&wt=javabin&version=2} {add=[3 (1438698658551824384)]} 0 13
[junit4:junit4] 2> 272195 T1553 C4544 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={test.distrib.skip.servers=http://127.0.0.1:55964/h_/collection1/&test.distrib.skip.servers=http://127.0.0.1:47592/h_/collection1/&wt=javabin&version=2} {add=[3]} 0 14
[junit4:junit4] 2> ASYNC NEW_CORE C4546 name=collection1 org.apache.solr.core.SolrCore@13b502f url=http://127.0.0.1:35478/h_/collection1 node=127.0.0.1:35478_h_ C4546_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:35478_h_, base_url=http://127.0.0.1:35478/h_, leader=true}
[junit4:junit4] 2> 272197 T1498 C4546 P35478 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272227 T1498 C4546 P35478 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 272227 T1498 C4546 P35478 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 272229 T1498 C4546 P35478 oass.SolrIndexSearcher.<init> Opening Searcher@148c22e main
[junit4:junit4] 2> 272230 T1498 C4546 P35478 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272231 T1508 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@148c22e main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272231 T1498 C4546 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 34
[junit4:junit4] 2> 272233 T1552 C4544 P47592 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44934/h_/collection1/, StdNode: http://127.0.0.1:55964/h_/collection1/, StdNode: http://127.0.0.1:47592/h_/collection1/, StdNode: http://127.0.0.1:43552/h_/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 272238 T1551 C4544 P47592 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272238 T1567 C4543 P43552 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272238 T1518 C4545 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4547 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4547_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 272240 T1535 C4547 P55964 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272287 T1567 C4543 P43552 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 272287 T1535 C4547 P55964 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 272287 T1518 C4545 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 272287 T1551 C4544 P47592 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 272289 T1551 C4544 P47592 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 272289 T1518 C4545 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 272288 T1535 C4547 P55964 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 272287 T1567 C4543 P43552 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 272291 T1551 C4544 P47592 oass.SolrIndexSearcher.<init> Opening Searcher@138076c main
[junit4:junit4] 2> 272292 T1518 C4545 P44934 oass.SolrIndexSearcher.<init> Opening Searcher@1c2cdac main
[junit4:junit4] 2> 272292 T1551 C4544 P47592 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272293 T1535 C4547 P55964 oass.SolrIndexSearcher.<init> Opening Searcher@1059691 main
[junit4:junit4] 2> 272293 T1526 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c2cdac main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272292 T1518 C4545 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272293 T1535 C4547 P55964 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272294 T1542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1059691 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 272293 T1567 C4543 P43552 oass.SolrIndexSearcher.<init> Opening Searcher@63ed68 main
[junit4:junit4] 2> 272293 T1559 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@138076c main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272295 T1567 C4543 P43552 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C4548 name=collection1 org.apache.solr.core.SolrCore@1cfdbfd url=http://127.0.0.1:55964/h_/collection1 node=127.0.0.1:55964_h_ C4548_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55964_h_, base_url=http://127.0.0.1:55964/h_}
[junit4:junit4] 2> 272295 T1535 C4548 P55964 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> ASYNC NEW_CORE C4549 name=collection1 org.apache.solr.core.SolrCore@26d025 url=http://127.0.0.1:44934/h_/collection1 node=127.0.0.1:44934_h_ C4549_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44934_h_, base_url=http://127.0.0.1:44934/h_, leader=true}
[junit4:junit4] 2> 272294 T1518 C4549 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 56
[junit4:junit4] 2> 272301 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@63ed68 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> ASYNC NEW_CORE C4550 name=collection1 org.apache.solr.core.SolrCore@1f1625 url=http://127.0.0.1:47592/h_/collection1 node=127.0.0.1:47592_h_ C4550_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47592_h_, base_url=http://127.0.0.1:47592/h_}
[junit4:junit4] 2> 272300 T1551 C4550 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 62
[junit4:junit4] 2> ASYNC NEW_CORE C4551 name=collection1 org.apache.solr.core.SolrCore@14f66aa url=http://127.0.0.1:43552/h_/collection1 node=127.0.0.1:43552_h_ C4551_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43552_h_, base_url=http://127.0.0.1:43552/h_}
[junit4:junit4] 2> 272302 T1567 C4551 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 64
[junit4:junit4] 2> 272303 T1552 C4550 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 70
[junit4:junit4] 2> 272303 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 272304 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 272305 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 272307 T1517 C4549 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 272309 T1536 C4548 P55964 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 272311 T1550 C4550 P47592 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 272313 T1569 C4551 P43552 oasc.SolrCore.execute [collection1] webapp=/h_ path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 272314 T1482 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 272319 T1553 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 272319 T1553 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 272322 T1520 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 272323 T1520 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 272324 T1520 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ START replicas=[http://127.0.0.1:55964/h_/collection1/, http://127.0.0.1:47592/h_/collection1/, http://127.0.0.1:43552/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 272330 T1570 C4551 P43552 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 272331 T1533 C4548 P55964 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=6
[junit4:junit4] 2> 272330 T1552 C4550 P47592 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 272331 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Received 5 versions from 127.0.0.1:43552/h_/collection1/
[junit4:junit4] 2> 272332 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Our versions are newer. ourLowThreshold=1438698658501492736 otherHigh=1438698658539241472
[junit4:junit4] 2> 272333 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Received 3 versions from 127.0.0.1:55964/h_/collection1/
[junit4:junit4] 2> 272333 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Our versions are newer. ourLowThreshold=1438698658501492736 otherHigh=1438698658528755712
[junit4:junit4] 2> 272334 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Received 5 versions from 127.0.0.1:47592/h_/collection1/
[junit4:junit4] 2> 272334 T1520 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ Our versions are newer. ourLowThreshold=1438698658501492736 otherHigh=1438698658539241472
[junit4:junit4] 2> 272334 T1520 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44934/h_ DONE. sync succeeded
[junit4:junit4] 2> 272335 T1520 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 272335 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: try and ask http://127.0.0.1:55964/h_/collection1/ to sync
[junit4:junit4] 2> 272336 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: try and ask http://127.0.0.1:47592/h_/collection1/ to sync
[junit4:junit4] 2> 272337 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: try and ask http://127.0.0.1:43552/h_/collection1/ to sync
[junit4:junit4] 2> 272338 T1535 C4548 P55964 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55964/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 272338 T1550 C4550 P47592 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47592/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 272338 T1567 C4551 P43552 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43552/h_ START replicas=[http://127.0.0.1:44934/h_/collection1/] nUpdates=100
[junit4:junit4] 2> 272340 T1519 C4549 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 272340 T1550 C4550 P47592 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:47592/h_ Received 5 versions from 127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 272341 T1518 C4549 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 272341 T1550 C4550 P47592 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:47592/h_ Our versions are newer. ourLowThreshold=1438698658501492736 otherHigh=1438698658539241472
[junit4:junit4] 2> 272340 T1519 C4549 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 272341 T1550 C4550 P47592 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47592/h_ DONE. sync succeeded
[junit4:junit4] 2> 272341 T1567 C4551 P43552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43552/h_ Received 5 versions from 127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 272342 T1550 C4550 P47592 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={sync=http://127.0.0.1:44934/h_/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 272342 T1535 C4548 P55964 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55964/h_ Received 5 versions from 127.0.0.1:44934/h_/collection1/
[junit4:junit4] 2> 272343 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: sync completed with http://127.0.0.1:47592/h_/collection1/
[junit4:junit4] 2> 272343 T1567 C4551 P43552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43552/h_ Our versions are newer. ourLowThreshold=1438698658501492736 otherHigh=1438698658539241472
[junit4:junit4] 2> 272344 T1535 C4548 P55964 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:55964/h_ Requesting updates from 127.0.0.1:44934/h_/collection1/n=2 versions=[1438698658551824384, 1438698658539241472]
[junit4:junit4] 2> 272344 T1567 C4551 P43552 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43552/h_ DONE. sync succeeded
[junit4:junit4] 2> 272345 T1567 C4551 P43552 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={sync=http://127.0.0.1:44934/h_/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7
[junit4:junit4] 2> 272346 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: sync completed with http://127.0.0.1:43552/h_/collection1/
[junit4:junit4] 2> 272347 T1517 C4549 P44934 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={distrib=false&getUpdates=1438698658551824384,1438698658539241472&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 272349 T1535 C4548 P55964 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438698658539241472), 3 (1438698658551824384)]} 0 2
[junit4:junit4] 2> 272350 T1535 C4548 P55964 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55964/h_ DONE. sync succeeded
[junit4:junit4] 2> 272350 T1535 C4548 P55964 oasc.SolrCore.execute [collection1] webapp=/h_ path=/get params={sync=http://127.0.0.1:44934/h_/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=12
[junit4:junit4] 2> 272350 T1520 oasc.SyncStrategy.syncToMe http://127.0.0.1:44934/h_/collection1/: sync completed with http://127.0.0.1:55964/h_/collection1/
[junit4:junit4] 2> 272351 T1520 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=29
[junit4:junit4] 2> 272351 T1553 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=33
[junit4:junit4] 2> 272352 T1482 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 272352 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 272353 T1482 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 272353 T1482 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C4552 name=collection1 org.apache.solr.core.SolrCore@13b502f url=http://127.0.0.1:35478/h_/collection1 node=127.0.0.1:35478_h_ C4552_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:35478_h_, base_url=http://127.0.0.1:35478/h_, leader=true}
[junit4:junit4] 2> 272355 T1500 C4552 P35478 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272407 T1500 C4552 P35478 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 272407 T1500 C4552 P35478 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 272408 T1500 C4552 P35478 oass.SolrIndexSearcher.<init> Opening Searcher@37030e main
[junit4:junit4] 2> 272408 T1500 C4552 P35478 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272409 T1508 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37030e main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272410 T1500 C4552 P35478 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> 272412 T1518 C4549 P44934 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44934/h_/collection1/, StdNode: http://127.0.0.1:55964/h_/collection1/, StdNode: http://127.0.0.1:47592/h_/collection1/, StdNode: http://127.0.0.1:43552/h_/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 272413 T1551 C4550 P47592 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272414 T1569 C4551 P43552 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272414 T1519 C4549 P44934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272413 T1536 C4548 P55964 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 272437 T1551 C4550 P47592 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 272437 T1551 C4550 P47592 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 272438 T1551 C4550 P47592 oass.SolrIndexSearcher.<init> Opening Searcher@1797e9b main
[junit4:junit4] 2> 272438 T1551 C4550 P47592 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272439 T1559 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1797e9b main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272440 T1551 C4550 P47592 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 27
[junit4:junit4] 2> 272457 T1569 C4551 P43552 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 272457 T1519 C4549 P44934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 272458 T1519 C4549 P44934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 272457 T1569 C4551 P43552 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 272458 T1519 C4549 P44934 oass.SolrIndexSearcher.<init> Opening Searcher@164e77 main
[junit4:junit4] 2> 272459 T1569 C4551 P43552 oass.SolrIndexSearcher.<init> Opening Searcher@10d00ed main
[junit4:junit4] 2> 272459 T1519 C4549 P44934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272459 T1569 C4551 P43552 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 272460 T1526 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@164e77 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272460 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10d00ed main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 272461 T1519 C4549 P44934 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 47
[junit4:junit4] 2> 272461 T1569 C4551 P43552 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 47
[junit4:junit4] 2> 272497 T1536 C4548 P55964 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372049938354/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/w
[...truncated too long message...]
1> "node_name":"127.0.0.1:47592_h_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:47592/h_"},
[junit4:junit4] 1> "5":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:43552_h_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:43552/h_",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:35478_h_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35478/h_",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:35478_h_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:43552_h_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:47592_h_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:44934_h_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89918664760098830-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89918664760098828-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89918664760098826-4-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:43552_h_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:43552/h_"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89918664760098819-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:35478_h_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35478/h_"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89918664760098826-127.0.0.1:47592_h_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89918664760098830-127.0.0.1:44934_h_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89918664760098828-127.0.0.1:43552_h_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89918664760098819-127.0.0.1:35478_h_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89918664760098819-127.0.0.1:35478_h_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=D3EE7114E39CA0F3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_CA -Dtests.timezone=Australia/Sydney -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 98.3s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:44934/h_/collection1lastClient and got 256 from http://127.0.0.1:47592/h_/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D3EE7114E39CA0F3:5208FF0C94C3C0CF]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 349161 T1482 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 98339 T1481 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 349278 T1557 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 349278 T1557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 349278 T1557 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=80), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=80)), sim=DefaultSimilarity, locale=en_CA, timezone=Australia/Sydney
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Sun Microsystems Inc. 1.6.0_45 (32-bit)/cpus=8,threads=1,free=24387520,total=93208576
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestStressVersions, TestSearchPerf, ClusterStateUpdateTest, TestExtendedDismaxParser, PluginInfoTest, SuggesterFSTTest, TestClassNameShortening, BadCopyFieldTest, SoftAutoCommitTest, SimpleFacetsTest, TestDocumentBuilder, TestDynamicFieldResource, TestSolrDeletionPolicy2, TestValueSourceCache, TestReplicationHandler, TestSolrIndexConfig, DateFieldTest, TestIBSimilarityFactory, RegexBoostProcessorTest, TestLFUCache, TestSchemaSimilarityResource, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, TestCharFilters, TestTrie, SchemaVersionSpecificBehaviorTest, TestXIncludeConfig, CoreAdminHandlerTest, BinaryUpdateRequestHandlerTest, TestCloudManagedSchema, ExternalFileFieldSortTest, JsonLoaderTest, DocumentBuilderTest, CachingDirectoryFactoryTest, SuggesterTest, AnalysisAfterCoreReloadTest, TestSchemaVersionResource, QueryResultKeyTest, IndexSchemaTest, TestPseudoReturnFields, UpdateRequestProcessorFactoryTest, CircularListTest, ZkControllerTest, TestRealTimeGet, PrimUtilsTest, TestSurroundQueryParser, SearchHandlerTest, TestSolrQueryParser, ResponseLogComponentTest, TestDocSet, TestFastLRUCache, TestSchemaResource, TestManagedSchema, ChaosMonkeyNothingIsSafeTest, ReturnFieldsTest, WordBreakSolrSpellCheckerTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 98.87s, 1 test, 1 failure <<< FAILURES!
[...truncated 684 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 47 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.6.0_45 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_25) - Build # 6213
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6213/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:55205/collection1lastClient and got 250 from http://127.0.0.1:34344/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:55205/collection1lastClient and got 250 from http://127.0.0.1:34344/collection1
at __randomizedtesting.SeedInfo.seed([855FC20E98EBFA85:4B94C16EFB49AB9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9367 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 63168 T192 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 63173 T192 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372060463474
[junit4:junit4] 2> 63174 T192 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 63175 T193 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 63275 T192 oasc.ZkTestServer.run start zk server on port:35619
[junit4:junit4] 2> 63276 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 63287 T199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bd058e name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 63288 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 63288 T192 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 63292 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 63293 T201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18a2f49 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 63293 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 63294 T192 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 63300 T192 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 63303 T192 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 63306 T192 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 63316 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 63317 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 63336 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 63337 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 63343 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 63343 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 63347 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 63348 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 63351 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 63352 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 63357 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 63358 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 63366 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 63367 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 63386 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 63386 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 63397 T192 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 63397 T192 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 63471 T192 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 63474 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34181
[junit4:junit4] 2> 63475 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 63475 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 63475 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706
[junit4:junit4] 2> 63476 T192 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/solr.xml
[junit4:junit4] 2> 63476 T192 oasc.CoreContainer.<init> New CoreContainer 24560448
[junit4:junit4] 2> 63476 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/'
[junit4:junit4] 2> 63477 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/'
[junit4:junit4] 2> 63521 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 63521 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 63522 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 63522 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 63522 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 63523 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 63523 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 63523 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 63523 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 63524 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 63528 T192 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 63528 T192 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35619/solr
[junit4:junit4] 2> 63529 T192 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 63530 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 63532 T212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@122d921 name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 63533 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 63535 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 63539 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 63540 T214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1138286 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 63541 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 63543 T192 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 63547 T192 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 63551 T192 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 63553 T192 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34181_
[junit4:junit4] 2> 63554 T192 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34181_
[junit4:junit4] 2> 63557 T192 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 63569 T192 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 63572 T192 oasc.Overseer.start Overseer (id=89919354534428675-127.0.0.1:34181_-n_0000000000) starting
[junit4:junit4] 2> 63578 T192 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 63583 T216 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 63584 T192 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 63587 T192 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 63590 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 63592 T215 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 63595 T217 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 63595 T217 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 65118 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 65119 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34181_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34181"}
[junit4:junit4] 2> 65120 T215 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 65120 T215 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 65128 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 65597 T217 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/collection1
[junit4:junit4] 2> 65597 T217 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 65598 T217 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 65598 T217 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 65600 T217 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/collection1/'
[junit4:junit4] 2> 65600 T217 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/collection1/lib/README' to classloader
[junit4:junit4] 2> 65601 T217 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 65631 T217 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 65670 T217 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 65672 T217 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 65678 T217 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 66180 T217 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 66180 T217 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 66181 T217 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 66188 T217 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 66190 T217 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 66207 T217 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 66211 T217 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 66215 T217 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 66216 T217 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 66216 T217 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 66216 T217 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 66218 T217 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 66218 T217 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 66218 T217 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 66224 T217 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372060463706/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/
[junit4:junit4] 2> 66225 T217 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@141794b
[junit4:junit4] 2> 66225 T217 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 66226 T217 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data
[junit4:junit4] 2> 66226 T217 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index/
[junit4:junit4] 2> 66226 T217 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 66227 T217 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index
[junit4:junit4] 2> 66266 T217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 66267 T217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 66269 T217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 66269 T217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 66270 T217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 66270 T217 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 66271 T217 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 66271 T217 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 66271 T217 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 66272 T217 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 66272 T217 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 66274 T217 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 66277 T217 oass.SolrIndexSearcher.<init> Opening Searcher@1536ee0 main
[junit4:junit4] 2> 66278 T217 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 66278 T217 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 66283 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1536ee0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 66285 T217 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 66285 T217 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34181 collection:control_collection shard:shard1
[junit4:junit4] 2> 66286 T217 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 66300 T217 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 66303 T217 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 66304 T217 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 66304 T217 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34181/collection1/
[junit4:junit4] 2> 66305 T217 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 66305 T217 oasc.SyncStrategy.syncToMe http://127.0.0.1:34181/collection1/ has no replicas
[junit4:junit4] 2> 66305 T217 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34181/collection1/
[junit4:junit4] 2> 66305 T217 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 66633 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 66669 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 66730 T217 oasc.ZkController.register We are http://127.0.0.1:34181/collection1/ and leader is http://127.0.0.1:34181/collection1/
[junit4:junit4] 2> 66730 T217 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34181
[junit4:junit4] 2> 66730 T217 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 66730 T217 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 66731 T217 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 66732 T217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 66734 T192 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 66734 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 66735 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 66738 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 66739 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 66741 T221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@609cb9 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 66741 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 66743 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 66746 T192 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 66814 T192 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 66816 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55205
[junit4:junit4] 2> 66817 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 66817 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 66818 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046
[junit4:junit4] 2> 66818 T192 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/solr.xml
[junit4:junit4] 2> 66818 T192 oasc.CoreContainer.<init> New CoreContainer 10266143
[junit4:junit4] 2> 66819 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/'
[junit4:junit4] 2> 66819 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/'
[junit4:junit4] 2> 66882 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 66882 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 66882 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 66883 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 66883 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 66883 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 66884 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 66884 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 66884 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 66885 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 66888 T192 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 66889 T192 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35619/solr
[junit4:junit4] 2> 66889 T192 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 66890 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 66892 T232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f2829e name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 66892 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 66894 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 66898 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 66900 T234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e84706 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 66900 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 66905 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 67909 T192 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55205_
[junit4:junit4] 2> 67910 T192 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55205_
[junit4:junit4] 2> 67913 T221 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 67913 T214 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 67913 T234 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 67913 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 67921 T235 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 67921 T235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 68199 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 68200 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34181__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34181_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34181"}
[junit4:junit4] 2> 68204 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55205_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55205"}
[junit4:junit4] 2> 68205 T215 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 68205 T215 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 68295 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 68295 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 68295 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 68922 T235 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/collection1
[junit4:junit4] 2> 68923 T235 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 68924 T235 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 68924 T235 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 68926 T235 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/collection1/'
[junit4:junit4] 2> 68927 T235 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/collection1/lib/README' to classloader
[junit4:junit4] 2> 68927 T235 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 68963 T235 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 68997 T235 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 68999 T235 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 69018 T235 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 69389 T235 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 69389 T235 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 69390 T235 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 69400 T235 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 69404 T235 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 69429 T235 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 69434 T235 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 69465 T235 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 69467 T235 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 69468 T235 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 69468 T235 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 69471 T235 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 69471 T235 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 69471 T235 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 69472 T235 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372060467046/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/
[junit4:junit4] 2> 69472 T235 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@141794b
[junit4:junit4] 2> 69473 T235 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 69473 T235 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1
[junit4:junit4] 2> 69474 T235 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index/
[junit4:junit4] 2> 69474 T235 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 69475 T235 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index
[junit4:junit4] 2> 69478 T235 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 69478 T235 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 69481 T235 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 69482 T235 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 69483 T235 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 69485 T235 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 69486 T235 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 69486 T235 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 69487 T235 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 69487 T235 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 69488 T235 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 69490 T235 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 69493 T235 oass.SolrIndexSearcher.<init> Opening Searcher@96cf52 main
[junit4:junit4] 2> 69493 T235 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 69493 T235 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 69499 T236 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@96cf52 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 69502 T235 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 69503 T235 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55205 collection:collection1 shard:shard1
[junit4:junit4] 2> 69504 T235 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 69513 T235 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 69516 T235 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 69517 T235 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 69517 T235 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 69517 T235 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 69517 T235 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/ has no replicas
[junit4:junit4] 2> 69518 T235 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 69518 T235 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 69803 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 69813 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 69813 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 69813 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 69826 T235 oasc.ZkController.register We are http://127.0.0.1:55205/collection1/ and leader is http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 69826 T235 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55205
[junit4:junit4] 2> 69827 T235 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 69827 T235 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 69827 T235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 69829 T235 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 69830 T192 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 69831 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 69832 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 69923 T192 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 69925 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38280
[junit4:junit4] 2> 69926 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 69926 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 69926 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137
[junit4:junit4] 2> 69927 T192 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/solr.xml
[junit4:junit4] 2> 69927 T192 oasc.CoreContainer.<init> New CoreContainer 21273044
[junit4:junit4] 2> 69927 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/'
[junit4:junit4] 2> 69928 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/'
[junit4:junit4] 2> 69971 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 69972 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 69972 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 69972 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 69973 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 69973 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 69973 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 69974 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 69974 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 69974 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 69978 T192 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 69978 T192 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35619/solr
[junit4:junit4] 2> 69979 T192 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 69979 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 69981 T248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a097e name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 69981 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 69983 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 69987 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 69992 T250 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a3da81 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 69992 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 69997 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 71001 T192 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38280_
[junit4:junit4] 2> 71003 T192 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38280_
[junit4:junit4] 2> 71006 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 71006 T214 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 71007 T250 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 71007 T234 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 71007 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 71007 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 71010 T221 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 71015 T251 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 71015 T251 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 71322 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 71323 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:55205__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:55205_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55205"}
[junit4:junit4] 2> 71327 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38280_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38280"}
[junit4:junit4] 2> 71327 T215 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 71327 T215 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 71331 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 71331 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 71331 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 71331 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 72017 T251 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/collection1
[junit4:junit4] 2> 72017 T251 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 72018 T251 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 72018 T251 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 72020 T251 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/collection1/'
[junit4:junit4] 2> 72020 T251 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/collection1/lib/README' to classloader
[junit4:junit4] 2> 72021 T251 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 72047 T251 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 72091 T251 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 72093 T251 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 72098 T251 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 72520 T251 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 72521 T251 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 72521 T251 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 72528 T251 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 72531 T251 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 72547 T251 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 72552 T251 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 72558 T251 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 72559 T251 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 72560 T251 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 72560 T251 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 72561 T251 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 72562 T251 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 72562 T251 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 72562 T251 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372060470137/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/
[junit4:junit4] 2> 72563 T251 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@141794b
[junit4:junit4] 2> 72564 T251 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 72564 T251 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2
[junit4:junit4] 2> 72565 T251 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index/
[junit4:junit4] 2> 72565 T251 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 72565 T251 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index
[junit4:junit4] 2> 72569 T251 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 72570 T251 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 72573 T251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 72574 T251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 72575 T251 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 72576 T251 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 72577 T251 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 72577 T251 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 72577 T251 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 72578 T251 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 72579 T251 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 72583 T251 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 72588 T251 oass.SolrIndexSearcher.<init> Opening Searcher@1025e3c main
[junit4:junit4] 2> 72589 T251 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 72589 T251 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 72597 T252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1025e3c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 72600 T251 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 72600 T251 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38280 collection:collection1 shard:shard1
[junit4:junit4] 2> 72605 T251 oasc.ZkController.register We are http://127.0.0.1:38280/collection1/ and leader is http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 72605 T251 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38280
[junit4:junit4] 2> 72605 T251 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 72606 T251 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C127 name=collection1 org.apache.solr.core.SolrCore@1280526 url=http://127.0.0.1:38280/collection1 node=127.0.0.1:38280_ C127_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38280_, base_url=http://127.0.0.1:38280}
[junit4:junit4] 2> 72606 T253 C127 P38280 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 72607 T253 C127 P38280 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 72607 T251 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 72607 T253 C127 P38280 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 72608 T253 C127 P38280 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 72608 T192 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 72609 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 72609 T253 C127 P38280 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 72609 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 72616 T227 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:38280__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 72682 T192 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 72684 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34344
[junit4:junit4] 2> 72685 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 72685 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 72685 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913
[junit4:junit4] 2> 72686 T192 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/solr.xml
[junit4:junit4] 2> 72686 T192 oasc.CoreContainer.<init> New CoreContainer 32444606
[junit4:junit4] 2> 72686 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/'
[junit4:junit4] 2> 72687 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/'
[junit4:junit4] 2> 72759 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 72759 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 72759 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 72760 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 72760 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 72760 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 72761 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 72761 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 72761 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 72762 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 72765 T192 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 72766 T192 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35619/solr
[junit4:junit4] 2> 72766 T192 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 72767 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 72768 T265 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7083c1 name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 72769 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 72771 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 72775 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 72776 T267 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15a8848 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 72776 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 72781 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 72839 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 72840 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:38280__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38280_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38280"}
[junit4:junit4] 2> 72845 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 72845 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 72845 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 72845 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 72845 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 73617 T227 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:38280__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 73617 T227 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:38280__collection1&state=recovering&nodeName=127.0.0.1:38280_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 73784 T192 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34344_
[junit4:junit4] 2> 73786 T192 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34344_
[junit4:junit4] 2> 73788 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 73788 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 73789 T234 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 73789 T267 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 73789 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 73789 T214 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 73790 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 73789 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 73790 T221 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 73791 T250 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 73795 T268 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 73795 T268 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 74350 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 74351 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34344_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34344"}
[junit4:junit4] 2> 74351 T215 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 74352 T215 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 74356 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 74356 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 74356 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 74356 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 74356 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 74797 T268 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/collection1
[junit4:junit4] 2> 74797 T268 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 74798 T268 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 74798 T268 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 74800 T268 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/collection1/'
[junit4:junit4] 2> 74801 T268 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/collection1/lib/README' to classloader
[junit4:junit4] 2> 74801 T268 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 74832 T268 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 74887 T268 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 74889 T268 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 74908 T268 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 75267 T268 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 75267 T268 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 75268 T268 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 75274 T268 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 75277 T268 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 75291 T268 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 75294 T268 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 75298 T268 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 75299 T268 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 75299 T268 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 75299 T268 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 75300 T268 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 75301 T268 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 75301 T268 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 75301 T268 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372060472913/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/
[junit4:junit4] 2> 75301 T268 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@141794b
[junit4:junit4] 2> 75302 T268 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 75302 T268 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3
[junit4:junit4] 2> 75303 T268 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index/
[junit4:junit4] 2> 75303 T268 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 75303 T268 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index
[junit4:junit4] 2> 75306 T268 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 75306 T268 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 75308 T268 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 75308 T268 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 75309 T268 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 75309 T268 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 75310 T268 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 75310 T268 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 75310 T268 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 75311 T268 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 75311 T268 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 75314 T268 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 75316 T268 oass.SolrIndexSearcher.<init> Opening Searcher@15e0164 main
[junit4:junit4] 2> 75317 T268 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 75317 T268 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 75323 T269 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15e0164 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 75326 T268 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 75326 T268 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34344 collection:collection1 shard:shard1
[junit4:junit4] 2> 75330 T268 oasc.ZkController.register We are http://127.0.0.1:34344/collection1/ and leader is http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 75330 T268 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34344
[junit4:junit4] 2> 75330 T268 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 75330 T268 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C128 name=collection1 org.apache.solr.core.SolrCore@e6646 url=http://127.0.0.1:34344/collection1 node=127.0.0.1:34344_ C128_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34344_, base_url=http://127.0.0.1:34344}
[junit4:junit4] 2> 75331 T270 C128 P34344 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 75332 T270 C128 P34344 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 75332 T268 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 75332 T270 C128 P34344 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 75332 T270 C128 P34344 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 75333 T192 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 75334 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 75334 T270 C128 P34344 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 75334 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 75339 T230 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:34344__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 75401 T192 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 75402 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44954
[junit4:junit4] 2> 75403 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 75403 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 75404 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638
[junit4:junit4] 2> 75404 T192 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/solr.xml
[junit4:junit4] 2> 75404 T192 oasc.CoreContainer.<init> New CoreContainer 1660885
[junit4:junit4] 2> 75405 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/'
[junit4:junit4] 2> 75405 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/'
[junit4:junit4] 2> 75448 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 75449 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 75449 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 75450 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 75450 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 75450 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 75451 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 75451 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 75451 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 75452 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 75455 T192 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 75455 T192 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35619/solr
[junit4:junit4] 2> 75456 T192 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 75456 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 75458 T282 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@746c52 name:ZooKeeperConnection Watcher:127.0.0.1:35619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 75458 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 75460 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 75463 T192 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 75464 T284 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13d0353 name:ZooKeeperConnection Watcher:127.0.0.1:35619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 75464 T192 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 75468 T192 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C129 name=collection1 org.apache.solr.core.SolrCore@1280526 url=http://127.0.0.1:38280/collection1 node=127.0.0.1:38280_ C129_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38280_, base_url=http://127.0.0.1:38280}
[junit4:junit4] 2> 75618 T253 C129 P38280 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55205/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 75619 T253 C129 P38280 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38280 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 75619 T253 C129 P38280 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 75619 T253 C129 P38280 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 75620 T253 C129 P38280 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 75620 T253 C129 P38280 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 75620 T253 C129 P38280 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 75620 T253 C129 P38280 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55205/collection1/. core=collection1
[junit4:junit4] 2> 75620 T253 C129 P38280 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C130 name=collection1 org.apache.solr.core.SolrCore@1b04710 url=http://127.0.0.1:55205/collection1 node=127.0.0.1:55205_ C130_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55205_, base_url=http://127.0.0.1:55205, leader=true}
[junit4:junit4] 2> 75626 T229 C130 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 75628 T227 C130 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 75630 T227 C130 P55205 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 75631 T227 C130 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 75635 T227 C130 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 75635 T227 C130 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 75636 T227 C130 P55205 oass.SolrIndexSearcher.<init> Opening Searcher@de896 realtime
[junit4:junit4] 2> 75636 T227 C130 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 75636 T227 C130 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 75637 T253 C129 P38280 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 75638 T253 C129 P38280 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 75640 T227 C130 P55205 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 75640 T227 C130 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 75641 T253 C129 P38280 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 75641 T253 C129 P38280 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 75641 T253 C129 P38280 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 75644 T229 C130 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 75644 T253 C129 P38280 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 75645 T253 C129 P38280 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945
[junit4:junit4] 2> 75645 T253 C129 P38280 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ca0d89 fullCopy=false
[junit4:junit4] 2> 75648 T228 C130 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C131 name=collection1 org.apache.solr.core.SolrCore@1280526 url=http://127.0.0.1:38280/collection1 node=127.0.0.1:38280_ C131_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38280_, base_url=http://127.0.0.1:38280}
[junit4:junit4] 2> 75663 T253 C131 P38280 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 75676 T253 C131 P38280 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 75676 T253 C131 P38280 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 75678 T253 C131 P38280 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 75678 T253 C131 P38280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 75679 T253 C131 P38280 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 75679 T253 C131 P38280 oass.SolrIndexSearcher.<init> Opening Searcher@1776447 main
[junit4:junit4] 2> 75680 T252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1776447 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 75681 T253 C131 P38280 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945;done=true>>]
[junit4:junit4] 2> 75681 T253 C131 P38280 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945
[junit4:junit4] 2> 75681 T253 C131 P38280 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index.20130624105435945
[junit4:junit4] 2> 75684 T253 C131 P38280 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 75684 T253 C131 P38280 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 75684 T253 C131 P38280 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 75684 T253 C131 P38280 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 75686 T253 C131 P38280 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 75862 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 75863 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34344__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34344_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34344"}
[junit4:junit4] 2> 75869 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:38280__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38280_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38280"}
[junit4:junit4] 2> 75895 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 75895 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 75896 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 75896 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 75896 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 75896 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 76340 T230 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:34344__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 76340 T230 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:34344__collection1&state=recovering&nodeName=127.0.0.1:34344_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 76471 T192 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44954_
[junit4:junit4] 2> 76473 T192 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44954_
[junit4:junit4] 2> 76475 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 76475 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 76476 T267 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76476 T214 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76476 T234 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76477 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 76476 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 76477 T284 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76477 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 76476 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 76480 T250 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76481 T221 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 76486 T287 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 76486 T287 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 77405 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 77406 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44954_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44954"}
[junit4:junit4] 2> 77406 T215 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 77406 T215 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 77411 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77411 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77412 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77412 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77412 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77412 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 77488 T287 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/collection1
[junit4:junit4] 2> 77488 T287 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 77489 T287 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 77489 T287 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 77491 T287 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/collection1/'
[junit4:junit4] 2> 77491 T287 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/collection1/lib/README' to classloader
[junit4:junit4] 2> 77491 T287 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 77520 T287 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 77551 T287 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 77552 T287 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 77557 T287 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 77948 T287 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 77948 T287 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 77949 T287 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 77955 T287 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 77957 T287 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 77975 T287 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 77980 T287 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 77984 T287 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 77985 T287 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 77986 T287 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 77986 T287 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 77987 T287 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 77987 T287 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 77988 T287 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 77988 T287 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372060475638/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/
[junit4:junit4] 2> 77989 T287 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@141794b
[junit4:junit4] 2> 77989 T287 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 77990 T287 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4
[junit4:junit4] 2> 77990 T287 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index/
[junit4:junit4] 2> 77991 T287 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 77991 T287 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index
[junit4:junit4] 2> 77995 T287 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 77995 T287 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 77998 T287 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 77999 T287 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 78000 T287 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 78001 T287 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 78002 T287 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 78002 T287 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 78003 T287 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 78004 T287 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 78005 T287 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 78008 T287 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 78011 T287 oass.SolrIndexSearcher.<init> Opening Searcher@b05dfd main
[junit4:junit4] 2> 78012 T287 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 78012 T287 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 78019 T288 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b05dfd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 78023 T287 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 78023 T287 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44954 collection:collection1 shard:shard1
[junit4:junit4] 2> 78026 T287 oasc.ZkController.register We are http://127.0.0.1:44954/collection1/ and leader is http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 78026 T287 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44954
[junit4:junit4] 2> 78026 T287 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 78027 T287 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C132 name=collection1 org.apache.solr.core.SolrCore@10a9854 url=http://127.0.0.1:44954/collection1 node=127.0.0.1:44954_ C132_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:44954_, base_url=http://127.0.0.1:44954}
[junit4:junit4] 2> 78027 T289 C132 P44954 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 78028 T289 C132 P44954 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 78028 T287 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 78028 T289 C132 P44954 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 78029 T289 C132 P44954 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 78029 T192 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 78030 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 78030 T289 C132 P44954 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 78030 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 78034 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 78036 T192 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 78036 T228 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:44954__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 78036 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 78038 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C133 name=collection1 org.apache.solr.core.SolrCore@e6646 url=http://127.0.0.1:34344/collection1 node=127.0.0.1:34344_ C133_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34344_, base_url=http://127.0.0.1:34344}
[junit4:junit4] 2> 78341 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55205/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 78342 T270 C133 P34344 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34344 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 78342 T270 C133 P34344 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 78342 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 78343 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 78343 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 78343 T270 C133 P34344 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 78343 T270 C133 P34344 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55205/collection1/. core=collection1
[junit4:junit4] 2> 78343 T270 C133 P34344 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C134 name=collection1 org.apache.solr.core.SolrCore@1b04710 url=http://127.0.0.1:55205/collection1 node=127.0.0.1:55205_ C134_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55205_, base_url=http://127.0.0.1:55205, leader=true}
[junit4:junit4] 2> 78345 T227 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 78349 T230 C134 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 78355 T230 C134 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 78355 T230 C134 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 78356 T230 C134 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 78356 T230 C134 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 78357 T270 C133 P34344 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 78357 T270 C133 P34344 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 78359 T227 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 78360 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 78360 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 78360 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 78363 T230 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 78363 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 78364 T270 C133 P34344 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664
[junit4:junit4] 2> 78364 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664 lockFactory=org.apache.lucene.store.NativeFSLockFactory@292ae5 fullCopy=false
[junit4:junit4] 2> 78367 T229 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 78369 T270 C133 P34344 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 78370 T270 C133 P34344 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 78371 T270 C133 P34344 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 78372 T270 C133 P34344 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 78372 T270 C133 P34344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 78373 T270 C133 P34344 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 78373 T270 C133 P34344 oass.SolrIndexSearcher.<init> Opening Searcher@188d7f1 main
[junit4:junit4] 2> 78375 T269 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@188d7f1 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 78375 T270 C133 P34344 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664;done=true>>]
[junit4:junit4] 2> 78376 T270 C133 P34344 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664
[junit4:junit4] 2> 78376 T270 C133 P34344 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index.20130624105438664
[junit4:junit4] 2> 78376 T270 C133 P34344 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 78376 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 78377 T270 C133 P34344 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 78377 T270 C133 P34344 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 78378 T270 C133 P34344 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 78920 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 78921 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44954__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44954_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44954"}
[junit4:junit4] 2> 78924 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34344__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34344_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34344"}
[junit4:junit4] 2> 78945 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 78945 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 78945 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 78945 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 78945 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 78945 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 79037 T228 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:44954__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 79037 T228 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:44954__collection1&state=recovering&nodeName=127.0.0.1:44954_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 79039 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 80041 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C132_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44954_, base_url=http://127.0.0.1:44954}
[junit4:junit4] 2> 81038 T289 C132 P44954 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55205/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 81039 T289 C132 P44954 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44954 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 81039 T289 C132 P44954 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 81039 T289 C132 P44954 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 81039 T289 C132 P44954 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 81040 T289 C132 P44954 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 81040 T289 C132 P44954 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 81040 T289 C132 P44954 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55205/collection1/. core=collection1
[junit4:junit4] 2> 81040 T289 C132 P44954 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 81042 T229 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 81043 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 81047 T230 C134 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 81055 T230 C134 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 81055 T230 C134 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 81056 T230 C134 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 81056 T230 C134 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 81057 T289 C132 P44954 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 81057 T289 C132 P44954 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 81060 T229 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 81060 T289 C132 P44954 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 81061 T289 C132 P44954 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 81061 T289 C132 P44954 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 81063 T227 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0
[junit4:junit4] 2> 81064 T289 C132 P44954 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 81065 T289 C132 P44954 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365
[junit4:junit4] 2> 81065 T289 C132 P44954 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e9b2ea fullCopy=false
[junit4:junit4] 2> 81069 T230 C134 P55205 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C135 name=collection1 org.apache.solr.core.SolrCore@10a9854 url=http://127.0.0.1:44954/collection1 node=127.0.0.1:44954_ C135_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44954_, base_url=http://127.0.0.1:44954}
[junit4:junit4] 2> 81087 T289 C135 P44954 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 81089 T289 C135 P44954 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 81089 T289 C135 P44954 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 81091 T289 C135 P44954 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 81091 T289 C135 P44954 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 81092 T289 C135 P44954 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 81092 T289 C135 P44954 oass.SolrIndexSearcher.<init> Opening Searcher@1df1f78 main
[junit4:junit4] 2> 81093 T288 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1df1f78 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 81094 T289 C135 P44954 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365;done=true>>]
[junit4:junit4] 2> 81094 T289 C135 P44954 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365
[junit4:junit4] 2> 81095 T289 C135 P44954 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index.20130624105441365
[junit4:junit4] 2> 81095 T289 C135 P44954 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 81095 T289 C135 P44954 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 81095 T289 C135 P44954 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 81096 T289 C135 P44954 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 81097 T289 C135 P44954 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 81956 T215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 81957 T215 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44954__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44954_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44954"}
[junit4:junit4] 2> 81961 T250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 81962 T214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 81962 T221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 81961 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 81962 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 81962 T234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 82045 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 82046 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C136 name=collection1 org.apache.solr.core.SolrCore@b5f627 url=http://127.0.0.1:34181/collection1 node=127.0.0.1:34181_ C136_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:34181_, base_url=http://127.0.0.1:34181, leader=true}
[junit4:junit4] 2> 82052 T209 C136 P34181 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 82054 T209 C136 P34181 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 82055 T209 C136 P34181 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 82085 T209 C136 P34181 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 82085 T209 C136 P34181 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 82086 T209 C136 P34181 oass.SolrIndexSearcher.<init> Opening Searcher@816c49 main
[junit4:junit4] 2> 82087 T209 C136 P34181 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 82087 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@816c49 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 82088 T209 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> C135_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44954_, base_url=http://127.0.0.1:44954}
[junit4:junit4] 2> 82099 T280 C135 P44954 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55205/collection1/, StdNode: http://127.0.0.1:38280/collection1/, StdNode: http://127.0.0.1:34344/collection1/, StdNode: http://127.0.0.1:44954/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 82106 T277 C135 P44954 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C137 name=collection1 org.apache.solr.core.SolrCore@1b04710 url=http://127.0.0.1:55205/collection1 node=127.0.0.1:55205_ C137_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55205_, base_url=http://127.0.0.1:55205, leader=true}
[junit4:junit4] 2> 82107 T229 C137 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C138 name=collection1 org.apache.solr.core.SolrCore@e6646 url=http://127.0.0.1:34344/collection1 node=127.0.0.1:34344_ C138_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34344_, base_url=http://127.0.0.1:34344}
[junit4:junit4] 2> 82110 T263 C138 P34344 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C139 name=collection1 org.apache.solr.core.SolrCore@1280526 url=http://127.0.0.1:38280/collection1 node=127.0.0.1:38280_ C139_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38280_, base_url=http://127.0.0.1:38280}
[junit4:junit4] 2> 82110 T244 C139 P38280 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 82117 T277 C135 P44954 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 82117 T277 C135 P44954 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 82118 T277 C135 P44954 oass.SolrIndexSearcher.<init> Opening Searcher@19d7e32 main
[junit4:junit4] 2> 82118 T277 C135 P44954 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 82119 T288 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19d7e32 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 82120 T277 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 82165 T263 C138 P34344 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 82165 T244 C139 P38280 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 82166 T244 C139 P38280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 82165 T229 C137 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 82167 T229 C137 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 82166 T263 C138 P34344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 82167 T229 C137 P55205 oass.SolrIndexSearcher.<init> Opening Searcher@130c3da main
[junit4:junit4] 2> 82167 T244 C139 P38280 oass.SolrIndexSearcher.<init> Opening Searcher@84e131 main
[junit4:junit4] 2> 82168 T229 C137 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 82169 T236 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@130c3da main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 82170 T252 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:302)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 82168 T263 C138 P34344 oass.SolrIndexSearcher.<init> Opening Searcher@1b9009 main
[junit4:junit4] 2> 82170 T244 C139 P38280 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 82170 T252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@84e131 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 82170 T229 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 82171 T244 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 61
[junit4:junit4] 2> 82171 T263 C138 P34344 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 82172 T269 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b9009 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 82173 T263 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 82173 T280 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 74
[junit4:junit4] 2> 82175 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 82181 T227 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 82183 T246 C139 P38280 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 82186 T260 C138 P34344 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 82189 T278 C135 P44954 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 84217 T208 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438709694611849216)} 0 8
[junit4:junit4] 2> 84229 T261 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438709694618140672&update.from=http://127.0.0.1:55205/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438709694618140672)} 0 1
[junit4:junit4] 2> 84231 T246 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438709694618140672&update.from=http://127.0.0.1:55205/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438709694618140672)} 0 1
[junit4:junit4] 2> 84229 T280 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438709694618140672&update.from=http://127.0.0.1:55205/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438709694618140672)} 0 1
[junit4:junit4] 2> 84232 T227 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438709694618140672)} 0 10
[junit4:junit4] 2> 84233 T263 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 14
[junit4:junit4] 2> 84239 T210 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438709694632820736)]} 0 3
[junit4:junit4] 2> 84257 T278 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438709694646452224)]} 0 3
[junit4:junit4] 2> 84258 T243 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438709694646452224)]} 0 4
[junit4:junit4] 2> 84258 T260 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438709694646452224)]} 0 4
[junit4:junit4] 2> 84259 T230 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34344/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438709694646452224)]} 0 10
[junit4:junit4] 2> 84260 T262 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 18
[junit4:junit4] 2> 84265 T209 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438709694661132288)]} 0 2
[junit4:junit4] 2> 84274 T244 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438709694666375168)]} 0 2
[junit4:junit4] 2> 84274 T279 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438709694666375168)]} 0 2
[junit4:junit4] 2> 84275 T261 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438709694666375168)]} 0 3
[junit4:junit4] 2> 84276 T229 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438709694666375168)]} 0 9
[junit4:junit4] 2> 84281 T207 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438709694677909504)]} 0 2
[junit4:junit4] 2> 84286 T227 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:38280/collection1/]
[junit4:junit4] 2> 84287 T227 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:38280/collection1/ against:[http://127.0.0.1:38280/collection1/] result:true
[junit4:junit4] 2> 84287 T227 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:34344/collection1/ against:[http://127.0.0.1:38280/collection1/] result:false
[junit4:junit4] 2> 84287 T227 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:44954/collection1/ against:[http://127.0.0.1:38280/collection1/] result:false
[junit4:junit4] 2> 84293 T260 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[2 (1438709694687346688)]} 0 2
[junit4:junit4] 2> 84294 T277 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[2 (1438709694687346688)]} 0 2
[junit4:junit4] 2> 84294 T227 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34344/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[2 (1438709694687346688)]} 0 8
[junit4:junit4] 2> 84295 T263 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[2]} 0 11
[junit4:junit4] 2> 84299 T210 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438709694697832448)]} 0 1
[junit4:junit4] 2> 84305 T230 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:38280/collection1/]
[junit4:junit4] 2> 84305 T230 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:38280/collection1/ against:[http://127.0.0.1:38280/collection1/] result:true
[junit4:junit4] 2> 84306 T230 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:34344/collection1/ against:[http://127.0.0.1:38280/collection1/] result:false
[junit4:junit4] 2> 84306 T230 C137 P55205 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:44954/collection1/ against:[http://127.0.0.1:38280/collection1/] result:false
[junit4:junit4] 2> 84312 T280 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[3 (1438709694707269632)]} 0 1
[junit4:junit4] 2> 84313 T261 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:55205/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[3 (1438709694707269632)]} 0 2
[junit4:junit4] 2> 84314 T230 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34344/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&wt=javabin&version=2} {add=[3 (1438709694707269632)]} 0 9
[junit4:junit4] 2> 84314 T262 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:38280/collection1/&test.distrib.skip.servers=http://127.0.0.1:34344/collection1/&wt=javabin&version=2} {add=[3]} 0 12
[junit4:junit4] 2> 84317 T209 C136 P34181 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84355 T209 C136 P34181 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 84355 T209 C136 P34181 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 84371 T209 C136 P34181 oass.SolrIndexSearcher.<init> Opening Searcher@104d8e2 main
[junit4:junit4] 2> 84371 T209 C136 P34181 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84372 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@104d8e2 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84373 T209 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 56
[junit4:junit4] 2> 84376 T228 C137 P55205 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55205/collection1/, StdNode: http://127.0.0.1:38280/collection1/, StdNode: http://127.0.0.1:34344/collection1/, StdNode: http://127.0.0.1:44954/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 84378 T229 C137 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84379 T260 C138 P34344 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84379 T278 C135 P44954 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84381 T245 C139 P38280 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84425 T245 C139 P38280 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 84425 T245 C139 P38280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 84425 T229 C137 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 84426 T229 C137 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 84427 T245 C139 P38280 oass.SolrIndexSearcher.<init> Opening Searcher@b8c4b5 main
[junit4:junit4] 2> 84427 T245 C139 P38280 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84428 T229 C137 P55205 oass.SolrIndexSearcher.<init> Opening Searcher@134ead0 main
[junit4:junit4] 2> 84428 T252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b8c4b5 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):c2)}
[junit4:junit4] 2> 84429 T229 C137 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84429 T245 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 84430 T236 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@134ead0 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84430 T229 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 52
[junit4:junit4] 2> 84435 T278 C135 P44954 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 84435 T278 C135 P44954 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 84435 T260 C138 P34344 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 84436 T260 C138 P34344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 84437 T278 C135 P44954 oass.SolrIndexSearcher.<init> Opening Searcher@1a62278 main
[junit4:junit4] 2> 84437 T278 C135 P44954 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84437 T260 C138 P34344 oass.SolrIndexSearcher.<init> Opening Searcher@1412a15 main
[junit4:junit4] 2> 84438 T260 C138 P34344 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84438 T288 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a62278 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84439 T278 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 60
[junit4:junit4] 2> 84439 T269 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1412a15 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84440 T260 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 61
[junit4:junit4] 2> 84440 T228 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 64
[junit4:junit4] 2> 84441 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 84442 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 84443 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 84446 T227 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 84448 T246 C139 P38280 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 84451 T263 C138 P34344 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 84454 T279 C135 P44954 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 84456 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 84462 T261 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 84463 T261 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 84467 T230 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 84468 T230 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55205/collection1/
[junit4:junit4] 2> 84468 T230 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55205 START replicas=[http://127.0.0.1:38280/collection1/, http://127.0.0.1:34344/collection1/, http://127.0.0.1:44954/collection1/] nUpdates=100
[junit4:junit4] 2> 84475 T260 C138 P34344 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 84475 T243 C139 P38280 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 84476 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Received 5 versions from 127.0.0.1:34344/collection1/
[junit4:junit4] 2> 84477 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Our versions are newer. ourLowThreshold=1438709694618140672 otherHigh=1438709694687346688
[junit4:junit4] 2> 84477 T280 C135 P44954 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 84477 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Received 3 versions from 127.0.0.1:38280/collection1/
[junit4:junit4] 2> 84478 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Our versions are newer. ourLowThreshold=1438709694618140672 otherHigh=1438709694666375168
[junit4:junit4] 2> 84478 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Received 5 versions from 127.0.0.1:44954/collection1/
[junit4:junit4] 2> 84478 T230 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:55205 Our versions are newer. ourLowThreshold=1438709694618140672 otherHigh=1438709694687346688
[junit4:junit4] 2> 84478 T230 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55205 DONE. sync succeeded
[junit4:junit4] 2> 84479 T230 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 84479 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: try and ask http://127.0.0.1:38280/collection1/ to sync
[junit4:junit4] 2> 84479 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: try and ask http://127.0.0.1:34344/collection1/ to sync
[junit4:junit4] 2> 84480 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: try and ask http://127.0.0.1:44954/collection1/ to sync
[junit4:junit4] 2> 84486 T244 C139 P38280 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38280 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 84487 T279 C135 P44954 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44954 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 84486 T263 C138 P34344 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34344 START replicas=[http://127.0.0.1:55205/collection1/] nUpdates=100
[junit4:junit4] 2> 84489 T228 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 84490 T229 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 84489 T227 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 84490 T263 C138 P34344 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34344 Received 5 versions from 127.0.0.1:55205/collection1/
[junit4:junit4] 2> 84491 T279 C135 P44954 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44954 Received 5 versions from 127.0.0.1:55205/collection1/
[junit4:junit4] 2> 84491 T263 C138 P34344 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34344 Our versions are newer. ourLowThreshold=1438709694618140672 otherHigh=1438709694687346688
[junit4:junit4] 2> 84491 T244 C139 P38280 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:38280 Received 5 versions from 127.0.0.1:55205/collection1/
[junit4:junit4] 2> 84492 T263 C138 P34344 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34344 DONE. sync succeeded
[junit4:junit4] 2> 84491 T279 C135 P44954 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44954 Our versions are newer. ourLowThreshold=1438709694618140672 otherHigh=1438709694687346688
[junit4:junit4] 2> 84492 T263 C138 P34344 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:55205/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=6
[junit4:junit4] 2> 84492 T244 C139 P38280 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:38280 Requesting updates from 127.0.0.1:55205/collection1/n=2 versions=[1438709694707269632, 1438709694687346688]
[junit4:junit4] 2> 84493 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: sync completed with http://127.0.0.1:34344/collection1/
[junit4:junit4] 2> 84493 T279 C135 P44954 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44954 DONE. sync succeeded
[junit4:junit4] 2> 84494 T279 C135 P44954 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:55205/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7
[junit4:junit4] 2> 84495 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: sync completed with http://127.0.0.1:44954/collection1/
[junit4:junit4] 2> 84496 T228 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438709694707269632,1438709694687346688&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 84507 T244 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438709694687346688), 3 (1438709694707269632)]} 0 10
[junit4:junit4] 2> 84508 T244 C139 P38280 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38280 DONE. sync succeeded
[junit4:junit4] 2> 84508 T244 C139 P38280 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:55205/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=22
[junit4:junit4] 2> 84509 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:55205/collection1/: sync completed with http://127.0.0.1:38280/collection1/
[junit4:junit4] 2> 84509 T230 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=42
[junit4:junit4] 2> 84510 T261 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=49
[junit4:junit4] 2> 84511 T192 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 84511 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 84512 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 84513 T192 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 84515 T207 C136 P34181 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84555 T207 C136 P34181 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 84555 T207 C136 P34181 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 84556 T207 C136 P34181 oass.SolrIndexSearcher.<init> Opening Searcher@1675e68 main
[junit4:junit4] 2> 84557 T207 C136 P34181 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84558 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1675e68 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84558 T207 C136 P34181 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 84561 T229 C137 P55205 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55205/collection1/, StdNode: http://127.0.0.1:38280/collection1/, StdNode: http://127.0.0.1:34344/collection1/, StdNode: http://127.0.0.1:44954/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 84564 T227 C137 P55205 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84564 T262 C138 P34344 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84564 T277 C135 P44954 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84564 T245 C139 P38280 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 84585 T227 C137 P55205 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 84586 T227 C137 P55205 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 84586 T227 C137 P55205 oass.SolrIndexSearcher.<init> Opening Searcher@97db9a main
[junit4:junit4] 2> 84587 T227 C137 P55205 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84588 T236 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@97db9a main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84588 T227 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 84605 T262 C138 P34344 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 84605 T277 C135 P44954 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 84606 T277 C135 P44954 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 84605 T262 C138 P34344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 84607 T277 C135 P44954 oass.SolrIndexSearcher.<init> Opening Searcher@a0fa74 main
[junit4:junit4] 2> 84607 T262 C138 P34344 oass.SolrIndexSearcher.<init> Opening Searcher@2146b2 main
[junit4:junit4] 2> 84607 T277 C135 P44954 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84608 T262 C138 P34344 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84609 T288 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a0fa74 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84609 T269 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2146b2 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 84609 T277 C135 P44954 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 45
[junit4:junit4] 2> 84610 T262 C138 P34344 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 46
[junit4:junit4] 2> 84645 T245 C139 P38280 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372060463474/jetty2/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 84646 T245 C139 P38280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 84647 T245 C139 P38280 oass.SolrIndexSearcher.<init> Opening Searcher@1360449 main
[junit4:junit4] 2> 84648 T245 C139 P38280 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 84649 T252 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1360449 main{StandardDirectoryReader(segments_5:5:nrt _0(4.4):c2 _1(4.4):c2)}
[junit4:junit4] 2> 84649 T245 C139 P38280 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 85
[junit4:junit4] 2> 84650 T229 C137 P55205 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 89
[junit4:junit4] 2> 84652 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 84655 T228 C137 P55205 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C140 name=collection1 org.apache.solr.core.SolrCore@1280526 url=http://127.0.0.1:38280/collection1 node=127.0.0.1:38280_ C140_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38280_, base_url=http://127.0.0.1:38280}
[junit4:junit4] 2> 84677 T243 C140 P38280 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=19
[junit4:junit4] 2> ASYNC NEW_CORE C141 name=collection1 org.apache.solr.core.SolrCore@e6646 url=http://127.0.0.1:34344/collection1 node=127.0.0.1:34344_ C141_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34344_, base_url=http://127.0.0.1:34344}
[junit4:junit4] 2> 84681 T263 C141 P34344 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C142 name=collection1 org.apache.solr.core.SolrCore@10a9854 url=http://127.0.0.1:44954/collection1 node=127.0.0.1:44954_ C142_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44954_, base_url=http://127.0.0.1:44954}
[junit4:junit4] 2> 84684 T278 C142 P44954 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 86686 T192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:127.0.0.1:55205__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:55205_","base_url":"http://127.0.0.1:55205","leader":"true"}
[junit4:junit4] 2> ASYNC NEW_CORE C143 name=collection1 org.apache.solr.core.SolrCore@1b04710 url=http://127.0.0.1:55205/collection1 node=127.0.0.1:55205_ C143_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55205_, base_url=http://127.0.0.1:55205, leader=true}
[junit4:junit4] 2> 86690 T230 C143 P55205 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:127.0.0.1:38280__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:38280_","base_url":"http://127.0.0.1:38280"}
[junit4:junit4] 2> 86693 T244 C140 P38280 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:127.0.0.1:34344__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:34344_","base_url":"http://127.0.0.1:34344"}
[junit4:junit4] 2> 86696 T263 C141 P34344 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:127.0.0.1:44954__collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:44954_","base_url":"http://127.0.0.1:44954"}
[junit4:junit4] 2> 86698 T279 C142 P44954 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&w
[...truncated too long message...]
"state":"down",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:38280_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:38280"},
[junit4:junit4] 1> "127.0.0.1:34344__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34344_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34344"},
[junit4:junit4] 1> "127.0.0.1:44954__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:44954_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:44954",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:34181__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:34181_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34181",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:34344_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:55205_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:34181_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:44954_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919354534428686-127.0.0.1:55205__collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919354534428684-127.0.0.1:44954__collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919354534428682-127.0.0.1:34344__collection1-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:44954_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:44954"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89919354534428675-127.0.0.1:34181__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34181_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34181"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89919354534428675-127.0.0.1:34181_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919354534428686-127.0.0.1:55205_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919354534428682-127.0.0.1:34344_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919354534428684-127.0.0.1:44954_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89919354534428675-127.0.0.1:34181_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=855FC20E98EBFA85 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hu_HU -Dtests.timezone=Israel -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 101s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:55205/collection1lastClient and got 250 from http://127.0.0.1:34344/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([855FC20E98EBFA85:4B94C16EFB49AB9]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 164550 T192 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 101385 T191 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 164651 T267 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 164652 T267 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 164652 T267 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 165738 T284 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 165739 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 165739 T284 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=hu_HU, timezone=Israel
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Oracle Corporation 1.7.0_25 (32-bit)/cpus=8,threads=1,free=48557208,total=103809024
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestQueryUtils, SuggesterTest, BadComponentTest, TestTrie, PrimUtilsTest, RegexBoostProcessorTest, TestFaceting, TestSurroundQueryParser, TermsComponentTest, QueryElevationComponentTest, TestCloudManagedSchema, TestPropInjectDefaults, SyncSliceTest]
[junit4:junit4] Completed on J0 in 102.66s, 1 test, 1 failure <<< FAILURES!
[...truncated 778 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 40 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 6212
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6212/
Java: 32bit/jdk1.6.0_45 -server -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:54803/_sp/collection1lastClient and got 253 from http://127.0.0.1:39516/_sp/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:54803/_sp/collection1lastClient and got 253 from http://127.0.0.1:39516/_sp/collection1
at __randomizedtesting.SeedInfo.seed([627E15E4F0145D12:E3989BFC874B3D2E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9072 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 719669 T2199 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_sp/
[junit4:junit4] 2> 719673 T2199 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1372055828133
[junit4:junit4] 2> 719673 T2199 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 719674 T2200 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 719774 T2199 oasc.ZkTestServer.run start zk server on port:48869
[junit4:junit4] 2> 719775 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 719777 T2206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c3fc1 name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 719778 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 719778 T2199 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 719781 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 719782 T2208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@153775d name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 719782 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 719783 T2199 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 719785 T2199 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 719787 T2199 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 719788 T2199 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 719791 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 719791 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 719794 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 719795 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 719798 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 719798 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 719800 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 719801 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 719803 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 719803 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 719806 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 719806 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 719808 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 719809 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 719811 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 719811 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 719813 T2199 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 719814 T2199 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 719884 T2199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 719886 T2199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40381
[junit4:junit4] 2> 719887 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 719887 T2199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 719888 T2199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277
[junit4:junit4] 2> 719888 T2199 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/solr.xml
[junit4:junit4] 2> 719888 T2199 oasc.CoreContainer.<init> New CoreContainer 19867050
[junit4:junit4] 2> 719889 T2199 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/'
[junit4:junit4] 2> 719889 T2199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/'
[junit4:junit4] 2> 719941 T2199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 719941 T2199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 719941 T2199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 719942 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 719942 T2199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 719942 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 719943 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 719943 T2199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 719943 T2199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 719944 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 719947 T2199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 719947 T2199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48869/solr
[junit4:junit4] 2> 719948 T2199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 719948 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 719950 T2219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18aef6 name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 719950 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 719952 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 719954 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 719957 T2221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a1c658 name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 719957 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 719959 T2199 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 719962 T2199 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 719964 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 719966 T2199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40381__sp
[junit4:junit4] 2> 719967 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40381__sp
[junit4:junit4] 2> 719969 T2199 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 719974 T2199 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 719976 T2199 oasc.Overseer.start Overseer (id=89919050752655363-127.0.0.1:40381__sp-n_0000000000) starting
[junit4:junit4] 2> 719978 T2199 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 719981 T2223 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 719981 T2199 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 719983 T2199 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 719984 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 719987 T2222 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 719990 T2224 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 719990 T2224 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 721490 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 721491 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40381__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40381/_sp"}
[junit4:junit4] 2> 721491 T2222 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 721491 T2222 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 721494 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 721991 T2224 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/collection1
[junit4:junit4] 2> 721991 T2224 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 721992 T2224 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 721992 T2224 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 721993 T2224 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/collection1/'
[junit4:junit4] 2> 721994 T2224 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/collection1/lib/README' to classloader
[junit4:junit4] 2> 721994 T2224 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 722020 T2224 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 722055 T2224 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 722056 T2224 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 722060 T2224 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 722434 T2224 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 722434 T2224 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 722435 T2224 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 722441 T2224 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 722444 T2224 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 722454 T2224 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 722457 T2224 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 722459 T2224 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 722460 T2224 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 722461 T2224 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 722461 T2224 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 722462 T2224 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 722462 T2224 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 722462 T2224 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 722462 T2224 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372055828277/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/
[junit4:junit4] 2> 722462 T2224 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a8d460
[junit4:junit4] 2> 722463 T2224 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 722463 T2224 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data
[junit4:junit4] 2> 722463 T2224 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index/
[junit4:junit4] 2> 722464 T2224 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 722464 T2224 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index
[junit4:junit4] 2> 722467 T2224 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 722467 T2224 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 722469 T2224 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 722469 T2224 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 722469 T2224 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 722470 T2224 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 722470 T2224 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 722470 T2224 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 722471 T2224 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 722471 T2224 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 722471 T2224 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 722473 T2224 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 722476 T2224 oass.SolrIndexSearcher.<init> Opening Searcher@106f58c main
[junit4:junit4] 2> 722476 T2224 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 722476 T2224 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 722480 T2225 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@106f58c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 722480 T2224 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 722481 T2224 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40381/_sp collection:control_collection shard:shard1
[junit4:junit4] 2> 722481 T2224 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 722487 T2224 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 722489 T2224 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 722489 T2224 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 722489 T2224 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40381/_sp/collection1/
[junit4:junit4] 2> 722489 T2224 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 722489 T2224 oasc.SyncStrategy.syncToMe http://127.0.0.1:40381/_sp/collection1/ has no replicas
[junit4:junit4] 2> 722489 T2224 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40381/_sp/collection1/
[junit4:junit4] 2> 722490 T2224 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 722998 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 723005 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 723046 T2224 oasc.ZkController.register We are http://127.0.0.1:40381/_sp/collection1/ and leader is http://127.0.0.1:40381/_sp/collection1/
[junit4:junit4] 2> 723046 T2224 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40381/_sp
[junit4:junit4] 2> 723046 T2224 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 723046 T2224 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 723046 T2224 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 723048 T2224 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 723049 T2199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 723049 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 723050 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 723052 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 723053 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 723055 T2228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e8fbf6 name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 723055 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 723056 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 723058 T2199 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 723123 T2199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 723124 T2199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:54803
[junit4:junit4] 2> 723125 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 723125 T2199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 723126 T2199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519
[junit4:junit4] 2> 723126 T2199 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/solr.xml
[junit4:junit4] 2> 723126 T2199 oasc.CoreContainer.<init> New CoreContainer 6927852
[junit4:junit4] 2> 723127 T2199 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/'
[junit4:junit4] 2> 723127 T2199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/'
[junit4:junit4] 2> 723177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 723177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 723177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 723178 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 723178 T2199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 723178 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 723179 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 723179 T2199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 723179 T2199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 723180 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 723184 T2199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 723184 T2199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48869/solr
[junit4:junit4] 2> 723185 T2199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 723186 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 723187 T2239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15c83e1 name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 723187 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 723189 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 723192 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 723193 T2241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f77af name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 723193 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 723196 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 724198 T2199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54803__sp
[junit4:junit4] 2> 724200 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54803__sp
[junit4:junit4] 2> 724203 T2228 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 724203 T2221 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 724203 T2241 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 724203 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 724207 T2242 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 724208 T2242 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 724510 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 724511 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40381__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40381/_sp"}
[junit4:junit4] 2> 724513 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:54803__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:54803/_sp"}
[junit4:junit4] 2> 724514 T2222 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 724514 T2222 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 724517 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 724517 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 724517 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 725209 T2242 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/collection1
[junit4:junit4] 2> 725209 T2242 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 725210 T2242 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 725210 T2242 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 725211 T2242 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/collection1/'
[junit4:junit4] 2> 725211 T2242 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/collection1/lib/README' to classloader
[junit4:junit4] 2> 725212 T2242 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 725238 T2242 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 725278 T2242 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 725279 T2242 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 725283 T2242 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 725661 T2242 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 725661 T2242 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 725662 T2242 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 725668 T2242 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 725671 T2242 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 725682 T2242 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 725685 T2242 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 725688 T2242 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 725690 T2242 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 725690 T2242 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 725690 T2242 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 725691 T2242 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 725691 T2242 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 725692 T2242 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 725692 T2242 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372055831519/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/
[junit4:junit4] 2> 725692 T2242 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a8d460
[junit4:junit4] 2> 725692 T2242 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 725693 T2242 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1
[junit4:junit4] 2> 725693 T2242 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index/
[junit4:junit4] 2> 725693 T2242 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 725694 T2242 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index
[junit4:junit4] 2> 725778 T2242 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 725778 T2242 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 725780 T2242 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 725780 T2242 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 725780 T2242 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 725781 T2242 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 725781 T2242 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 725782 T2242 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 725782 T2242 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 725782 T2242 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 725783 T2242 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 725784 T2242 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 725787 T2242 oass.SolrIndexSearcher.<init> Opening Searcher@16025d main
[junit4:junit4] 2> 725787 T2242 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 725788 T2242 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 725791 T2243 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16025d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 725793 T2242 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 725793 T2242 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54803/_sp collection:collection1 shard:shard1
[junit4:junit4] 2> 725794 T2242 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 725799 T2242 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 725801 T2242 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 725801 T2242 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 725801 T2242 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 725801 T2242 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 725801 T2242 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/ has no replicas
[junit4:junit4] 2> 725801 T2242 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 725802 T2242 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 726022 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 726031 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 726031 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 726031 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 726057 T2242 oasc.ZkController.register We are http://127.0.0.1:54803/_sp/collection1/ and leader is http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 726057 T2242 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54803/_sp
[junit4:junit4] 2> 726057 T2242 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 726057 T2242 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 726057 T2242 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 726060 T2242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 726061 T2199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 726061 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 726062 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 726128 T2199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 726130 T2199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48046
[junit4:junit4] 2> 726130 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 726131 T2199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 726131 T2199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524
[junit4:junit4] 2> 726131 T2199 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/solr.xml
[junit4:junit4] 2> 726132 T2199 oasc.CoreContainer.<init> New CoreContainer 4960411
[junit4:junit4] 2> 726132 T2199 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/'
[junit4:junit4] 2> 726132 T2199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/'
[junit4:junit4] 2> 726175 T2199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 726175 T2199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 726175 T2199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 726176 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 726176 T2199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 726176 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 726177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 726177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 726177 T2199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 726178 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 726180 T2199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 726181 T2199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48869/solr
[junit4:junit4] 2> 726181 T2199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 726182 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726183 T2255 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65d302 name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726184 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726185 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 726188 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726189 T2257 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc4227 name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726189 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726193 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 727196 T2199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48046__sp
[junit4:junit4] 2> 727197 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48046__sp
[junit4:junit4] 2> 727199 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 727200 T2221 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 727200 T2228 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 727200 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727200 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727201 T2257 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 727201 T2241 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 727206 T2258 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 727206 T2258 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 727546 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 727547 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:54803__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:54803/_sp"}
[junit4:junit4] 2> 727550 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48046__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48046/_sp"}
[junit4:junit4] 2> 727550 T2222 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 727550 T2222 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 727552 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727552 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727552 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 727552 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 728207 T2258 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/collection1
[junit4:junit4] 2> 728207 T2258 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 728208 T2258 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 728208 T2258 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 728209 T2258 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/collection1/'
[junit4:junit4] 2> 728210 T2258 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/collection1/lib/README' to classloader
[junit4:junit4] 2> 728210 T2258 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 728250 T2258 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 728283 T2258 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 728284 T2258 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 728288 T2258 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 728686 T2258 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 728687 T2258 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 728688 T2258 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 728694 T2258 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 728697 T2258 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 728709 T2258 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 728712 T2258 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 728714 T2258 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 728715 T2258 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 728715 T2258 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 728715 T2258 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 728716 T2258 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 728716 T2258 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 728717 T2258 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 728717 T2258 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372055834524/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/
[junit4:junit4] 2> 728717 T2258 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a8d460
[junit4:junit4] 2> 728717 T2258 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 728718 T2258 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2
[junit4:junit4] 2> 728718 T2258 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index/
[junit4:junit4] 2> 728718 T2258 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 728718 T2258 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index
[junit4:junit4] 2> 728721 T2258 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 728722 T2258 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 728723 T2258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 728723 T2258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 728724 T2258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 728724 T2258 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 728725 T2258 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 728725 T2258 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 728725 T2258 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 728726 T2258 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 728726 T2258 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 728728 T2258 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 728730 T2258 oass.SolrIndexSearcher.<init> Opening Searcher@1370385 main
[junit4:junit4] 2> 728731 T2258 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 728731 T2258 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 728736 T2259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1370385 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 728738 T2258 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 728738 T2258 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48046/_sp collection:collection1 shard:shard1
[junit4:junit4] 2> 728741 T2258 oasc.ZkController.register We are http://127.0.0.1:48046/_sp/collection1/ and leader is http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 728742 T2258 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48046/_sp
[junit4:junit4] 2> 728742 T2258 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 728742 T2258 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4886 name=collection1 org.apache.solr.core.SolrCore@1594bd8 url=http://127.0.0.1:48046/_sp/collection1 node=127.0.0.1:48046__sp C4886_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48046__sp, base_url=http://127.0.0.1:48046/_sp}
[junit4:junit4] 2> 728742 T2260 C4886 P48046 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 728743 T2260 C4886 P48046 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 728743 T2258 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 728744 T2260 C4886 P48046 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 728744 T2260 C4886 P48046 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 728744 T2199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 728745 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 728745 T2260 C4886 P48046 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 728745 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 728749 T2235 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 728814 T2199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 728816 T2199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39516
[junit4:junit4] 2> 728817 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 728817 T2199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 728817 T2199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208
[junit4:junit4] 2> 728818 T2199 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/solr.xml
[junit4:junit4] 2> 728818 T2199 oasc.CoreContainer.<init> New CoreContainer 19772986
[junit4:junit4] 2> 728818 T2199 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/'
[junit4:junit4] 2> 728819 T2199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/'
[junit4:junit4] 2> 728873 T2199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 728873 T2199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 728873 T2199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 728874 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 728874 T2199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 728874 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 728875 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 728875 T2199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 728875 T2199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 728875 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 728878 T2199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 728879 T2199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48869/solr
[junit4:junit4] 2> 728879 T2199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 728880 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 728882 T2272 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19ac684 name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 728882 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 728884 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 728888 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 728889 T2274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1312a4b name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 728890 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 728894 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 729057 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 729057 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48046__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48046/_sp"}
[junit4:junit4] 2> 729060 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729060 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729060 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729061 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729061 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729750 T2235 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 729750 T2235 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:48046__sp&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 729897 T2199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39516__sp
[junit4:junit4] 2> 729898 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39516__sp
[junit4:junit4] 2> 729900 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 729901 T2257 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 729901 T2221 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 729901 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 729901 T2228 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 729901 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 729901 T2274 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 729902 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 729901 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 729902 T2241 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 729906 T2275 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 729906 T2275 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 730565 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 730566 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39516__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39516/_sp"}
[junit4:junit4] 2> 730566 T2222 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 730566 T2222 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 730570 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 730570 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 730570 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 730570 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 730571 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 730908 T2275 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/collection1
[junit4:junit4] 2> 730908 T2275 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 730909 T2275 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 730909 T2275 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 730910 T2275 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/collection1/'
[junit4:junit4] 2> 730911 T2275 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/collection1/lib/README' to classloader
[junit4:junit4] 2> 730911 T2275 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 730937 T2275 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 730970 T2275 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 730971 T2275 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 730975 T2275 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 731356 T2275 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 731357 T2275 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 731357 T2275 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 731364 T2275 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 731367 T2275 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 731377 T2275 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 731380 T2275 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 731383 T2275 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 731384 T2275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 731384 T2275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 731384 T2275 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 731385 T2275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 731385 T2275 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 731385 T2275 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 731386 T2275 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372055837208/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/
[junit4:junit4] 2> 731386 T2275 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a8d460
[junit4:junit4] 2> 731386 T2275 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 731387 T2275 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3
[junit4:junit4] 2> 731387 T2275 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index/
[junit4:junit4] 2> 731387 T2275 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 731387 T2275 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index
[junit4:junit4] 2> 731390 T2275 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 731390 T2275 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 731392 T2275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 731392 T2275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 731393 T2275 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 731393 T2275 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 731394 T2275 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 731394 T2275 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 731394 T2275 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 731394 T2275 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 731395 T2275 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 731396 T2275 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 731399 T2275 oass.SolrIndexSearcher.<init> Opening Searcher@17b7a9a main
[junit4:junit4] 2> 731400 T2275 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 731400 T2275 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 731405 T2276 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17b7a9a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 731407 T2275 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 731407 T2275 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39516/_sp collection:collection1 shard:shard1
[junit4:junit4] 2> 731411 T2275 oasc.ZkController.register We are http://127.0.0.1:39516/_sp/collection1/ and leader is http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 731411 T2275 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39516/_sp
[junit4:junit4] 2> 731411 T2275 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 731412 T2275 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4887 name=collection1 org.apache.solr.core.SolrCore@6f1f47 url=http://127.0.0.1:39516/_sp/collection1 node=127.0.0.1:39516__sp C4887_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:39516__sp, base_url=http://127.0.0.1:39516/_sp}
[junit4:junit4] 2> 731412 T2277 C4887 P39516 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 731413 T2277 C4887 P39516 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 731413 T2275 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 731413 T2277 C4887 P39516 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 731413 T2277 C4887 P39516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 731414 T2199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 731414 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 731414 T2277 C4887 P39516 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 731414 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 731418 T2237 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 731481 T2199 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 731483 T2199 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45724
[junit4:junit4] 2> 731483 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 731484 T2199 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 731484 T2199 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877
[junit4:junit4] 2> 731484 T2199 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/solr.xml
[junit4:junit4] 2> 731485 T2199 oasc.CoreContainer.<init> New CoreContainer 16278027
[junit4:junit4] 2> 731485 T2199 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/'
[junit4:junit4] 2> 731485 T2199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/'
[junit4:junit4] 2> 731536 T2199 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 731536 T2199 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 731537 T2199 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 731537 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 731537 T2199 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 731538 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 731538 T2199 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 731538 T2199 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 731539 T2199 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 731539 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 731542 T2199 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 731542 T2199 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48869/solr
[junit4:junit4] 2> 731543 T2199 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 731543 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 731545 T2289 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@175f94e name:ZooKeeperConnection Watcher:127.0.0.1:48869 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 731545 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 731547 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 731550 T2199 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 731551 T2291 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a66223 name:ZooKeeperConnection Watcher:127.0.0.1:48869/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 731552 T2199 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 731555 T2199 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C4888 name=collection1 org.apache.solr.core.SolrCore@1594bd8 url=http://127.0.0.1:48046/_sp/collection1 node=127.0.0.1:48046__sp C4888_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48046__sp, base_url=http://127.0.0.1:48046/_sp}
[junit4:junit4] 2> 731751 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54803/_sp/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 731752 T2260 C4888 P48046 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48046/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 731752 T2260 C4888 P48046 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 731752 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 731752 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 731753 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 731753 T2260 C4888 P48046 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 731753 T2260 C4888 P48046 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54803/_sp/collection1/. core=collection1
[junit4:junit4] 2> 731753 T2260 C4888 P48046 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4889 name=collection1 org.apache.solr.core.SolrCore@171ad4e url=http://127.0.0.1:54803/_sp/collection1 node=127.0.0.1:54803__sp C4889_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54803__sp, base_url=http://127.0.0.1:54803/_sp, leader=true}
[junit4:junit4] 2> 731758 T2234 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 731761 T2235 C4889 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 731763 T2235 C4889 P54803 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 731763 T2235 C4889 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 731776 T2235 C4889 P54803 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 731777 T2235 C4889 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 731777 T2235 C4889 P54803 oass.SolrIndexSearcher.<init> Opening Searcher@1c2c8e realtime
[junit4:junit4] 2> 731777 T2235 C4889 P54803 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 731778 T2235 C4889 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> 731778 T2260 C4888 P48046 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 731778 T2260 C4888 P48046 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 731780 T2234 C4889 P54803 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 731780 T2234 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 731780 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 731780 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 731781 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 731782 T2235 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 731783 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 731783 T2260 C4888 P48046 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243
[junit4:junit4] 2> 731784 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243 lockFactory=org.apache.lucene.store.NativeFSLockFactory@464856; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 731786 T2236 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 731787 T2260 C4888 P48046 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 731797 T2260 C4888 P48046 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 731797 T2260 C4888 P48046 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 731799 T2260 C4888 P48046 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 731799 T2260 C4888 P48046 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 731799 T2260 C4888 P48046 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 731800 T2260 C4888 P48046 oass.SolrIndexSearcher.<init> Opening Searcher@61f79c main
[junit4:junit4] 2> 731801 T2259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@61f79c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 731801 T2260 C4888 P48046 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243;done=true>>]
[junit4:junit4] 2> 731801 T2260 C4888 P48046 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243
[junit4:junit4] 2> 731801 T2260 C4888 P48046 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index.20130624093720243
[junit4:junit4] 2> 731802 T2260 C4888 P48046 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 731802 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 731802 T2260 C4888 P48046 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 731802 T2260 C4888 P48046 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 731804 T2260 C4888 P48046 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 732074 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 732075 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39516__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39516/_sp"}
[junit4:junit4] 2> 732078 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48046__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48046/_sp"}
[junit4:junit4] 2> 732096 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732097 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732096 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732096 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732097 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732097 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732419 T2237 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 732419 T2237 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:39516__sp&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 732558 T2199 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45724__sp
[junit4:junit4] 2> 732560 T2199 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45724__sp
[junit4:junit4] 2> 732561 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 732562 T2221 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732562 T2257 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732562 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 732562 T2228 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732562 T2291 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732563 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 732562 T2274 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732563 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 732563 T2241 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 732562 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 732563 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 732568 T2294 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 732568 T2294 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 733604 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 733605 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45724__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45724/_sp"}
[junit4:junit4] 2> 733605 T2222 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 733605 T2222 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 733608 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 733608 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 733608 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 733608 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 733608 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 733608 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> ASYNC NEW_CORE C4890 name=collection1 org.apache.solr.core.SolrCore@6f1f47 url=http://127.0.0.1:39516/_sp/collection1 node=127.0.0.1:39516__sp C4890_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:39516__sp, base_url=http://127.0.0.1:39516/_sp}
[junit4:junit4] 2> 734420 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54803/_sp/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 734420 T2277 C4890 P39516 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39516/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 734420 T2277 C4890 P39516 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 734421 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 734421 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 734421 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 734421 T2277 C4890 P39516 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 734421 T2277 C4890 P39516 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54803/_sp/collection1/. core=collection1
[junit4:junit4] 2> 734422 T2277 C4890 P39516 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 734422 T2237 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 734425 T2234 C4889 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 734436 T2234 C4889 P54803 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 734437 T2234 C4889 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 734437 T2234 C4889 P54803 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 734437 T2234 C4889 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 734438 T2277 C4890 P39516 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 734438 T2277 C4890 P39516 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 734440 T2237 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 734440 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 734441 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 734441 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 734443 T2236 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 734443 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 734444 T2277 C4890 P39516 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904
[junit4:junit4] 2> 734444 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1347205; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 734447 T2234 C4889 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 734448 T2277 C4890 P39516 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 734450 T2277 C4890 P39516 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 734450 T2277 C4890 P39516 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 734452 T2277 C4890 P39516 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 734452 T2277 C4890 P39516 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 734453 T2277 C4890 P39516 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 734453 T2277 C4890 P39516 oass.SolrIndexSearcher.<init> Opening Searcher@eac987 main
[junit4:junit4] 2> 734454 T2276 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@eac987 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 734454 T2277 C4890 P39516 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904;done=true>>]
[junit4:junit4] 2> 734455 T2277 C4890 P39516 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904
[junit4:junit4] 2> 734455 T2277 C4890 P39516 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index.20130624093722904
[junit4:junit4] 2> 734455 T2277 C4890 P39516 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 734455 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 734455 T2277 C4890 P39516 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 734456 T2277 C4890 P39516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 734476 T2277 C4890 P39516 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 734570 T2294 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/collection1
[junit4:junit4] 2> 734570 T2294 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 734571 T2294 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 734571 T2294 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 734572 T2294 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/collection1/'
[junit4:junit4] 2> 734572 T2294 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/collection1/lib/README' to classloader
[junit4:junit4] 2> 734573 T2294 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 734609 T2294 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 734643 T2294 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 734644 T2294 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 734648 T2294 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 735035 T2294 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 735036 T2294 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 735036 T2294 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 735042 T2294 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 735045 T2294 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 735056 T2294 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 735059 T2294 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 735062 T2294 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 735063 T2294 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 735063 T2294 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 735063 T2294 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 735064 T2294 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 735064 T2294 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 735064 T2294 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 735065 T2294 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372055839877/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/
[junit4:junit4] 2> 735065 T2294 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a8d460
[junit4:junit4] 2> 735065 T2294 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 735066 T2294 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4
[junit4:junit4] 2> 735066 T2294 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index/
[junit4:junit4] 2> 735066 T2294 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 735066 T2294 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index
[junit4:junit4] 2> 735070 T2294 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 735070 T2294 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 735072 T2294 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 735072 T2294 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 735073 T2294 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 735073 T2294 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 735074 T2294 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 735074 T2294 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 735075 T2294 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 735075 T2294 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 735076 T2294 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 735077 T2294 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 735080 T2294 oass.SolrIndexSearcher.<init> Opening Searcher@811700 main
[junit4:junit4] 2> 735080 T2294 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 735081 T2294 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 735086 T2297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@811700 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 735088 T2294 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 735089 T2294 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45724/_sp collection:collection1 shard:shard1
[junit4:junit4] 2> 735091 T2294 oasc.ZkController.register We are http://127.0.0.1:45724/_sp/collection1/ and leader is http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 735092 T2294 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45724/_sp
[junit4:junit4] 2> 735092 T2294 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 735092 T2294 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4891 name=collection1 org.apache.solr.core.SolrCore@199aed1 url=http://127.0.0.1:45724/_sp/collection1 node=127.0.0.1:45724__sp C4891_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:45724__sp, base_url=http://127.0.0.1:45724/_sp}
[junit4:junit4] 2> 735093 T2298 C4891 P45724 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 735093 T2294 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735093 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 735094 T2298 C4891 P45724 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 735094 T2298 C4891 P45724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 735094 T2199 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 735095 T2199 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 735095 T2298 C4891 P45724 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 735095 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 735101 T2236 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 735101 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735102 T2199 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 735103 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 735103 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735113 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735114 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39516__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39516/_sp"}
[junit4:junit4] 2> 735117 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45724__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45724/_sp"}
[junit4:junit4] 2> 735120 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 735120 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 735120 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 735120 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 735120 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 735120 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 736102 T2236 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 736102 T2236 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:45724__sp&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 736105 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 737107 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C4891_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:45724__sp, base_url=http://127.0.0.1:45724/_sp}
[junit4:junit4] 2> 738103 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54803/_sp/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 738103 T2298 C4891 P45724 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45724/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 738103 T2298 C4891 P45724 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 738103 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 738104 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 738104 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 738104 T2298 C4891 P45724 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 738104 T2298 C4891 P45724 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54803/_sp/collection1/. core=collection1
[junit4:junit4] 2> 738104 T2298 C4891 P45724 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4892 name=collection1 org.apache.solr.core.SolrCore@171ad4e url=http://127.0.0.1:54803/_sp/collection1 node=127.0.0.1:54803__sp C4892_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54803__sp, base_url=http://127.0.0.1:54803/_sp, leader=true}
[junit4:junit4] 2> 738105 T2236 C4892 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 738107 T2235 C4892 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 738108 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 738116 T2235 C4892 P54803 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 738117 T2235 C4892 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 738117 T2235 C4892 P54803 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 738117 T2235 C4892 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 738118 T2298 C4891 P45724 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 738118 T2298 C4891 P45724 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 738120 T2234 C4892 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 738120 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 738120 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 738120 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 738122 T2235 C4892 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0
[junit4:junit4] 2> 738123 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 738124 T2298 C4891 P45724 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583
[junit4:junit4] 2> 738124 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@121b876; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 738126 T2237 C4892 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 738127 T2298 C4891 P45724 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 738129 T2298 C4891 P45724 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 738129 T2298 C4891 P45724 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 738131 T2298 C4891 P45724 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 738131 T2298 C4891 P45724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 738132 T2298 C4891 P45724 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 738132 T2298 C4891 P45724 oass.SolrIndexSearcher.<init> Opening Searcher@8042c8 main
[junit4:junit4] 2> 738133 T2297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8042c8 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 738133 T2298 C4891 P45724 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583;done=true>>]
[junit4:junit4] 2> 738133 T2298 C4891 P45724 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583
[junit4:junit4] 2> 738133 T2298 C4891 P45724 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index.20130624093726583
[junit4:junit4] 2> 738134 T2298 C4891 P45724 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 738134 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 738134 T2298 C4891 P45724 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 738134 T2298 C4891 P45724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 738157 T2298 C4891 P45724 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 739110 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 739630 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 739631 T2222 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45724__sp",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45724/_sp"}
[junit4:junit4] 2> 739634 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739634 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739634 T2228 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739634 T2241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739634 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739634 T2257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740112 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 740113 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C4893 name=collection1 org.apache.solr.core.SolrCore@e5b230 url=http://127.0.0.1:40381/_sp/collection1 node=127.0.0.1:40381__sp C4893_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:40381__sp, base_url=http://127.0.0.1:40381/_sp, leader=true}
[junit4:junit4] 2> 740119 T2215 C4893 P40381 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 740121 T2215 C4893 P40381 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 740121 T2215 C4893 P40381 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 740126 T2215 C4893 P40381 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 740127 T2215 C4893 P40381 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 740127 T2215 C4893 P40381 oass.SolrIndexSearcher.<init> Opening Searcher@1080c4a main
[junit4:junit4] 2> 740128 T2215 C4893 P40381 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 740128 T2225 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1080c4a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 740129 T2215 C4893 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> ASYNC NEW_CORE C4894 name=collection1 org.apache.solr.core.SolrCore@6f1f47 url=http://127.0.0.1:39516/_sp/collection1 node=127.0.0.1:39516__sp C4894_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39516__sp, base_url=http://127.0.0.1:39516/_sp}
[junit4:junit4] 2> 740136 T2270 C4894 P39516 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:54803/_sp/collection1/, StdNode: http://127.0.0.1:48046/_sp/collection1/, StdNode: http://127.0.0.1:39516/_sp/collection1/, StdNode: http://127.0.0.1:45724/_sp/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 740139 T2236 C4892 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 740141 T2269 C4894 P39516 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C4891_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:45724__sp, base_url=http://127.0.0.1:45724/_sp}
[junit4:junit4] 2> 740144 T2284 C4891 P45724 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4895 name=collection1 org.apache.solr.core.SolrCore@1594bd8 url=http://127.0.0.1:48046/_sp/collection1 node=127.0.0.1:48046__sp C4895_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:48046__sp, base_url=http://127.0.0.1:48046/_sp}
[junit4:junit4] 2> 740147 T2251 C4895 P48046 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 740156 T2236 C4892 P54803 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 740157 T2236 C4892 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 740157 T2236 C4892 P54803 oass.SolrIndexSearcher.<init> Opening Searcher@dd9613 main
[junit4:junit4] 2> 740158 T2236 C4892 P54803 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 740158 T2243 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@dd9613 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 740159 T2236 C4892 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 740176 T2284 C4891 P45724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 740176 T2269 C4894 P39516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 740177 T2269 C4894 P39516 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 740177 T2284 C4891 P45724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 740178 T2269 C4894 P39516 oass.SolrIndexSearcher.<init> Opening Searcher@9e96b0 main
[junit4:junit4] 2> 740178 T2284 C4891 P45724 oass.SolrIndexSearcher.<init> Opening Searcher@12b49ca main
[junit4:junit4] 2> 740178 T2269 C4894 P39516 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 740179 T2276 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9e96b0 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 740180 T2297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12b49ca main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 740180 T2269 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4] 2> 740179 T2284 C4891 P45724 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 740180 T2284 C4891 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> 740186 T2251 C4895 P48046 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 740187 T2251 C4895 P48046 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 740188 T2251 C4895 P48046 oass.SolrIndexSearcher.<init> Opening Searcher@1a35dd9 main
[junit4:junit4] 2> 740188 T2251 C4895 P48046 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 740189 T2259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a35dd9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 740189 T2251 C4895 P48046 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 42
[junit4:junit4] 2> 740190 T2270 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 54
[junit4:junit4] 2> 740191 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 740193 T2234 C4892 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 740194 T2253 C4895 P48046 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 740195 T2268 C4894 P39516 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 740196 T2286 C4891 P45724 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 742199 T2215 C4893 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438704835659563008)} 0 1
[junit4:junit4] 2> 742206 T2286 C4891 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={update.distrib=FROMLEADER&_version_=-1438704835664805888&update.from=http://127.0.0.1:54803/_sp/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438704835664805888)} 0 1
[junit4:junit4] 2> 742206 T2268 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={update.distrib=FROMLEADER&_version_=-1438704835664805888&update.from=http://127.0.0.1:54803/_sp/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438704835664805888)} 0 1
[junit4:junit4] 2> 742206 T2253 C4895 P48046 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={update.distrib=FROMLEADER&_version_=-1438704835664805888&update.from=http://127.0.0.1:54803/_sp/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438704835664805888)} 0 1
[junit4:junit4] 2> 742207 T2234 C4892 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438704835664805888)} 0 5
[junit4:junit4] 2> 742208 T2270 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 7
[junit4:junit4] 2> 742211 T2216 C4893 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[0 (1438704835672145920)]} 0 1
[junit4:junit4] 2> 742217 T2287 C4891 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438704835676340224)]} 0 1
[junit4:junit4] 2> 742217 T2250 C4895 P48046 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438704835676340224)]} 0 1
[junit4:junit4] 2> 742217 T2269 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438704835676340224)]} 0 1
[junit4:junit4] 2> 742218 T2235 C4892 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:39516/_sp/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438704835676340224)]} 0 4
[junit4:junit4] 2> 742219 T2267 C4894 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[0]} 0 6
[junit4:junit4] 2> 742221 T2217 C4893 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[1 (1438704835682631680)]} 0 1
[junit4:junit4] 2> 742227 T2284 C4891 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438704835686825984)]} 0 1
[junit4:junit4] 2> 742227 T2252 C4895 P48046 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438704835686825984)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C4896 name=collection1 org.apache.solr.core.SolrCore@6f1f47 url=http://127.0.0.1:39516/_sp/collection1 node=127.0.0.1:39516__sp C4896_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39516__sp, base_url=http://127.0.0.1:39516/_sp}
[junit4:junit4] 2> 742227 T2270 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438704835686825984)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C4897 name=collection1 org.apache.solr.core.SolrCore@171ad4e url=http://127.0.0.1:54803/_sp/collection1 node=127.0.0.1:54803__sp C4897_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54803__sp, base_url=http://127.0.0.1:54803/_sp, leader=true}
[junit4:junit4] 2> 742239 T2237 C4897 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:39516/_sp/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438704835686825984)]} 0 15
[junit4:junit4] 2> 742240 T2268 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[1]} 0 17
[junit4:junit4] 2> ASYNC NEW_CORE C4898 name=collection1 org.apache.solr.core.SolrCore@e5b230 url=http://127.0.0.1:40381/_sp/collection1 node=127.0.0.1:40381__sp C4898_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:40381__sp, base_url=http://127.0.0.1:40381/_sp, leader=true}
[junit4:junit4] 2> 742242 T2214 C4898 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[2 (1438704835705700352)]} 0 1
[junit4:junit4] 2> 742243 T2236 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:48046/_sp/collection1/]
[junit4:junit4] 2> 742244 T2236 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:48046/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:true
[junit4:junit4] 2> 742244 T2236 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39516/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:false
[junit4:junit4] 2> 742244 T2236 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:45724/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:false
[junit4:junit4] 2> 742247 T2269 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[2 (1438704835708846080)]} 0 0
[junit4:junit4] 2> ASYNC NEW_CORE C4899 name=collection1 org.apache.solr.core.SolrCore@199aed1 url=http://127.0.0.1:45724/_sp/collection1 node=127.0.0.1:45724__sp C4899_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:45724__sp, base_url=http://127.0.0.1:45724/_sp}
[junit4:junit4] 2> 742247 T2285 C4899 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[2 (1438704835708846080)]} 0 0
[junit4:junit4] 2> 742248 T2236 C4897 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[2 (1438704835708846080)]} 0 5
[junit4:junit4] 2> 742250 T2215 C4898 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={wt=javabin&version=2} {add=[3 (1438704835714088960)]} 0 0
[junit4:junit4] 2> 742254 T2234 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:48046/_sp/collection1/]
[junit4:junit4] 2> 742254 T2234 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:48046/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:true
[junit4:junit4] 2> 742254 T2234 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39516/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:false
[junit4:junit4] 2> 742255 T2234 C4897 P54803 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:45724/_sp/collection1/ against:[http://127.0.0.1:48046/_sp/collection1/] result:false
[junit4:junit4] 2> 742257 T2270 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[3 (1438704835719331840)]} 0 0
[junit4:junit4] 2> 742257 T2286 C4899 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:54803/_sp/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[3 (1438704835719331840)]} 0 0
[junit4:junit4] 2> 742258 T2234 C4897 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={distrib.from=http://127.0.0.1:39516/_sp/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&wt=javabin&version=2} {add=[3 (1438704835719331840)]} 0 4
[junit4:junit4] 2> 742258 T2267 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={test.distrib.skip.servers=http://127.0.0.1:48046/_sp/collection1/&test.distrib.skip.servers=http://127.0.0.1:39516/_sp/collection1/&wt=javabin&version=2} {add=[3]} 0 6
[junit4:junit4] 2> 742260 T2216 C4898 P40381 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742286 T2216 C4898 P40381 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 742287 T2216 C4898 P40381 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 742291 T2216 C4898 P40381 oass.SolrIndexSearcher.<init> Opening Searcher@15081fd main
[junit4:junit4] 2> 742291 T2216 C4898 P40381 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742292 T2225 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15081fd main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 742292 T2216 C4898 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> 742294 T2287 C4899 P45724 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:54803/_sp/collection1/, StdNode: http://127.0.0.1:48046/_sp/collection1/, StdNode: http://127.0.0.1:39516/_sp/collection1/, StdNode: http://127.0.0.1:45724/_sp/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 742298 T2268 C4896 P39516 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742298 T2235 C4897 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4900 name=collection1 org.apache.solr.core.SolrCore@1594bd8 url=http://127.0.0.1:48046/_sp/collection1 node=127.0.0.1:48046__sp C4900_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:48046__sp, base_url=http://127.0.0.1:48046/_sp}
[junit4:junit4] 2> 742298 T2251 C4900 P48046 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742298 T2284 C4899 P45724 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742346 T2235 C4897 P54803 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fcfd9c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 742347 T2235 C4897 P54803 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 742351 T2235 C4897 P54803 oass.SolrIndexSearcher.<init> Opening Searcher@13c9766 main
[junit4:junit4] 2> 742351 T2235 C4897 P54803 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742352 T2243 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13c9766 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 742353 T2235 C4897 P54803 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> 742356 T2251 C4900 P48046 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3a6e9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 742357 T2251 C4900 P48046 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 742360 T2251 C4900 P48046 oass.SolrIndexSearcher.<init> Opening Searcher@5ada15 main
[junit4:junit4] 2> 742361 T2251 C4900 P48046 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742361 T2259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ada15 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 742362 T2251 C4900 P48046 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 64
[junit4:junit4] 2> 742386 T2284 C4899 P45724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@173d11a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 742387 T2284 C4899 P45724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 742390 T2284 C4899 P45724 oass.SolrIndexSearcher.<init> Opening Searcher@607e58 main
[junit4:junit4] 2> 742391 T2284 C4899 P45724 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742391 T2297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@607e58 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 742392 T2284 C4899 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 94
[junit4:junit4] 2> 742416 T2268 C4896 P39516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 742417 T2268 C4896 P39516 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 742420 T2268 C4896 P39516 oass.SolrIndexSearcher.<init> Opening Searcher@48a47 main
[junit4:junit4] 2> 742420 T2268 C4896 P39516 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742421 T2276 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@48a47 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 742421 T2268 C4896 P39516 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 123
[junit4:junit4] 2> 742422 T2287 C4899 P45724 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 128
[junit4:junit4] 2> 742422 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 742423 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 742423 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 742425 T2237 C4897 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 742427 T2253 C4900 P48046 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 742429 T2269 C4896 P39516 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 742431 T2285 C4899 P45724 oasc.SolrCore.execute [collection1] webapp=/_sp path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 742432 T2199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 742437 T2270 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 742438 T2270 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 742442 T2236 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 742442 T2236 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 742443 T2236 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp START replicas=[http://127.0.0.1:48046/_sp/collection1/, http://127.0.0.1:39516/_sp/collection1/, http://127.0.0.1:45724/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 742444 T2250 C4900 P48046 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742444 T2286 C4899 P45724 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742444 T2267 C4896 P39516 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742444 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Received 3 versions from 127.0.0.1:48046/_sp/collection1/
[junit4:junit4] 2> 742445 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Our versions are newer. ourLowThreshold=1438704835664805888 otherHigh=1438704835686825984
[junit4:junit4] 2> 742445 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Received 5 versions from 127.0.0.1:45724/_sp/collection1/
[junit4:junit4] 2> 742445 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Our versions are newer. ourLowThreshold=1438704835664805888 otherHigh=1438704835708846080
[junit4:junit4] 2> 742446 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Received 5 versions from 127.0.0.1:39516/_sp/collection1/
[junit4:junit4] 2> 742446 T2236 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp Our versions are newer. ourLowThreshold=1438704835664805888 otherHigh=1438704835708846080
[junit4:junit4] 2> 742446 T2236 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:54803/_sp DONE. sync succeeded
[junit4:junit4] 2> 742446 T2236 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 742447 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: try and ask http://127.0.0.1:48046/_sp/collection1/ to sync
[junit4:junit4] 2> 742447 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: try and ask http://127.0.0.1:39516/_sp/collection1/ to sync
[junit4:junit4] 2> 742447 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: try and ask http://127.0.0.1:45724/_sp/collection1/ to sync
[junit4:junit4] 2> 742448 T2252 C4900 P48046 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48046/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 742448 T2269 C4896 P39516 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39516/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 742448 T2284 C4899 P45724 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45724/_sp START replicas=[http://127.0.0.1:54803/_sp/collection1/] nUpdates=100
[junit4:junit4] 2> 742449 T2234 C4897 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742450 T2269 C4896 P39516 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39516/_sp Received 5 versions from 127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 742450 T2237 C4897 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742450 T2235 C4897 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742450 T2252 C4900 P48046 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:48046/_sp Received 5 versions from 127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 742450 T2269 C4896 P39516 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39516/_sp Our versions are newer. ourLowThreshold=1438704835664805888 otherHigh=1438704835708846080
[junit4:junit4] 2> 742451 T2284 C4899 P45724 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:45724/_sp Received 5 versions from 127.0.0.1:54803/_sp/collection1/
[junit4:junit4] 2> 742451 T2252 C4900 P48046 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:48046/_sp Requesting updates from 127.0.0.1:54803/_sp/collection1/n=2 versions=[1438704835719331840, 1438704835708846080]
[junit4:junit4] 2> 742452 T2284 C4899 P45724 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:45724/_sp Our versions are newer. ourLowThreshold=1438704835664805888 otherHigh=1438704835708846080
[junit4:junit4] 2> 742452 T2284 C4899 P45724 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45724/_sp DONE. sync succeeded
[junit4:junit4] 2> 742451 T2269 C4896 P39516 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39516/_sp DONE. sync succeeded
[junit4:junit4] 2> 742452 T2284 C4899 P45724 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={sync=http://127.0.0.1:54803/_sp/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 742453 T2269 C4896 P39516 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={sync=http://127.0.0.1:54803/_sp/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 742453 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: sync completed with http://127.0.0.1:45724/_sp/collection1/
[junit4:junit4] 2> 742453 T2234 C4897 P54803 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={distrib=false&getUpdates=1438704835719331840,1438704835708846080&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 742454 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: sync completed with http://127.0.0.1:39516/_sp/collection1/
[junit4:junit4] 2> 742456 T2252 C4900 P48046 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438704835708846080), 3 (1438704835719331840)]} 0 2
[junit4:junit4] 2> 742456 T2252 C4900 P48046 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48046/_sp DONE. sync succeeded
[junit4:junit4] 2> 742457 T2252 C4900 P48046 oasc.SolrCore.execute [collection1] webapp=/_sp path=/get params={sync=http://127.0.0.1:54803/_sp/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=9
[junit4:junit4] 2> 742457 T2236 oasc.SyncStrategy.syncToMe http://127.0.0.1:54803/_sp/collection1/: sync completed with http://127.0.0.1:48046/_sp/collection1/
[junit4:junit4] 2> 742458 T2236 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=15
[junit4:junit4] 2> 742458 T2270 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=22
[junit4:junit4] 2> 742458 T2199 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 742459 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 742460 T2199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 742460 T2199 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 742462 T2217 C4898 P40381 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742486 T2217 C4898 P40381 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18476fc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 742487 T2217 C4898 P40381 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 742487 T2217 C4898 P40381 oass.SolrIndexSearcher.<init> Opening Searcher@bd434c main
[junit4:junit4] 2> 742488 T2217 C4898 P40381 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 742489 T2225 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bd434c main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 742489 T2217 C4898 P40381 oasup.LogUpdateProcessor.finish [collection1] webapp=/_sp path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 27
[junit4:junit4] 2> 742491 T2267 C4896 P39516 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:54803/_sp/collection1/, StdNode: http://127.0.0.1:48046/_sp/collection1/, StdNode: http://127.0.0.1:39516/_sp/collection1/, StdNode: http://127.0.0.1:45724/_sp/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 742493 T2251 C4900 P48046 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742493 T2268 C4896 P39516 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742493 T2237 C4897 P54803 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742494 T2287 C4899 P45724 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 742516 T2268 C4896 P39516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1372055828133/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@abe449; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 742517 T22
[...truncated too long message...]
"base_url":"http://127.0.0.1:40381/_sp",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:40381__sp (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:54803__sp (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:45724__sp (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:39516__sp (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919050752655374-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919050752655372-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89919050752655370-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:39516__sp",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39516/_sp"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89919050752655363-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:40381__sp",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40381/_sp"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89919050752655374-127.0.0.1:54803__sp-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919050752655372-127.0.0.1:45724__sp-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919050752655370-127.0.0.1:39516__sp-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89919050752655363-127.0.0.1:40381__sp-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89919050752655363-127.0.0.1:40381__sp-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=627E15E4F0145D12 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_MX -Dtests.timezone=Europe/Athens -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 99.8s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:54803/_sp/collection1lastClient and got 253 from http://127.0.0.1:39516/_sp/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([627E15E4F0145D12:E3989BFC874B3D2E]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 819476 T2199 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 99811 T2198 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 819549 T2274 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 819549 T2274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 819549 T2274 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 820637 T2291 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 820638 T2291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 820638 T2291 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=Lucene41WithOrds), rnd_b=MockFixedIntBlock(blockSize=1983), intDefault=PostingsFormat(name=Lucene41WithOrds), id=MockFixedIntBlock(blockSize=1983), timestamp=PostingsFormat(name=Lucene41WithOrds), a_t=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=Lucene41WithOrds), multiDefault=PostingsFormat(name=Asserting), a_si=PostingsFormat(name=Asserting)}, docValues:{}, sim=DefaultSimilarity, locale=es_MX, timezone=Europe/Athens
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Sun Microsystems Inc. 1.6.0_45 (32-bit)/cpus=8,threads=1,free=57153088,total=167780352
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReload, TestNumberUtils, PathHierarchyTokenizerFactoryTest, TestArbitraryIndexDir, ConvertedLegacyTest, DocValuesMultiTest, PreAnalyzedUpdateProcessorTest, UpdateParamsTest, TestUtils, TestWordDelimiterFilterFactory, DocumentAnalysisRequestHandlerTest, TestCollationField, TestUpdate, TestElisionMultitermQuery, TestCoreDiscovery, TestShardHandlerFactory, DisMaxRequestHandlerTest, DOMUtilTest, TestComponentsName, TestCSVResponseWriter, DefaultValueUpdateProcessorTest, SpellCheckComponentTest, TestStressReorder, HardAutoCommitTest, EchoParamsTest, SolrPluginUtilsTest, XmlUpdateRequestHandlerTest, TestSolrQueryParserResource, TestSolr4Spatial, TestSchemaNameResource, MultiTermTest, TestFieldTypeCollectionResource, TestBinaryField, TestFoldingMultitermQuery, TestMaxScoreQueryParser, TestMergePolicyConfig, SolrCmdDistributorTest, ShardRoutingCustomTest, TestCopyFieldCollectionResource, TestStressLucene, TestAnalyzedSuggestions, SampleTest, OpenCloseCoreStressTest, TestDefaultSearchFieldResource, DirectUpdateHandlerTest, UniqFieldsUpdateProcessorFactoryTest, SolrIndexConfigTest, TestDistributedSearch, ChaosMonkeySafeLeaderTest, RequestHandlersTest, FullSolrCloudDistribCmdsTest, OutputWriterTest, TestOmitPositions, TestManagedSchemaFieldResource, TestRandomFaceting, URLClassifyProcessorTest, TestCodecSupport, PingRequestHandlerTest, TestFieldTypeResource, TestDefaultSimilarityFactory, TestAtomicUpdateErrorCases, TestHashPartitioner, JSONWriterTest, TestQuerySenderNoQuery, OpenExchangeRatesOrgProviderTest, ClusterStateTest, TestSolrXMLSerializer, TestQueryTypes, TestPostingsSolrHighlighter, CoreContainerCoreInitFailuresTest, BasicDistributedZk2Test, SyncSliceTest]
[junit4:junit4] Completed on J1 in 101.11s, 1 test, 1 failure <<< FAILURES!
[...truncated 448 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 40 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.6.0_45 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure