You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/20 00:40:14 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 6144
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6144/
Java: 32bit/jdk1.6.0_45 -server -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:37650/collection1lastClient and got 258 from http://127.0.0.1:60040/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:37650/collection1lastClient and got 258 from http://127.0.0.1:60040/collection1
at __randomizedtesting.SeedInfo.seed([EC67CF367991BAF3:6D81412E0ECEDACF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9065 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 521249 T2033 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 521258 T2033 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1371680914174
[junit4:junit4] 2> 521259 T2033 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 521261 T2034 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 521361 T2033 oasc.ZkTestServer.run start zk server on port:51660
[junit4:junit4] 2> 521362 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521494 T2040 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1af5d0c name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521494 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521495 T2033 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 521500 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521502 T2042 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e6d362 name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521502 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521503 T2033 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 521508 T2033 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 521510 T2033 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 521512 T2033 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 521513 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 521514 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 521518 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 521518 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 521553 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 521553 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 521555 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 521556 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 521558 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 521558 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 521561 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 521561 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 521564 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 521565 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 521587 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 521587 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 521595 T2033 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 521596 T2033 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 521785 T2033 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 521788 T2033 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42500
[junit4:junit4] 2> 521789 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 521790 T2033 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 521790 T2033 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527
[junit4:junit4] 2> 521791 T2033 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/solr.xml
[junit4:junit4] 2> 521791 T2033 oasc.CoreContainer.<init> New CoreContainer 8441598
[junit4:junit4] 2> 521792 T2033 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/'
[junit4:junit4] 2> 521792 T2033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/'
[junit4:junit4] 2> 521850 T2033 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 521850 T2033 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 521851 T2033 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 521851 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 521852 T2033 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 521852 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 521853 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 521854 T2033 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 521854 T2033 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 521855 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 521858 T2033 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 521859 T2033 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51660/solr
[junit4:junit4] 2> 521860 T2033 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 521866 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521866 T2053 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20a318 name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521867 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521870 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 521877 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521879 T2055 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18858ba name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521879 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521883 T2033 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 521891 T2033 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 521894 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 521896 T2033 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42500_
[junit4:junit4] 2> 521897 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42500_
[junit4:junit4] 2> 521899 T2033 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 521939 T2033 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 521941 T2033 oasc.Overseer.start Overseer (id=89894480391569411-127.0.0.1:42500_-n_0000000000) starting
[junit4:junit4] 2> 521944 T2033 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 521957 T2057 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 521958 T2033 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 521967 T2033 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 521969 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 521972 T2056 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 521978 T2058 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 521979 T2058 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 523475 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 523476 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42500_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42500"}
[junit4:junit4] 2> 523476 T2056 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 523476 T2056 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 523480 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 523980 T2058 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/collection1
[junit4:junit4] 2> 523980 T2058 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 523981 T2058 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 523981 T2058 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 523982 T2058 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/collection1/'
[junit4:junit4] 2> 523983 T2058 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/collection1/lib/README' to classloader
[junit4:junit4] 2> 523983 T2058 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 524023 T2058 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 524073 T2058 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 524085 T2058 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 524091 T2058 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 524626 T2058 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 524632 T2058 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 524635 T2058 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 524659 T2058 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 524662 T2058 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 524666 T2058 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 524667 T2058 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 524667 T2058 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 524667 T2058 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 524668 T2058 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 524668 T2058 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 524668 T2058 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 524669 T2058 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371680914527/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/
[junit4:junit4] 2> 524669 T2058 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a0b53e
[junit4:junit4] 2> 524669 T2058 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 524670 T2058 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data
[junit4:junit4] 2> 524670 T2058 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index/
[junit4:junit4] 2> 524670 T2058 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 524671 T2058 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index
[junit4:junit4] 2> 524675 T2058 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 524675 T2058 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 524677 T2058 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 524677 T2058 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 524678 T2058 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 524679 T2058 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 524679 T2058 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 524679 T2058 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 524680 T2058 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 524680 T2058 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 524681 T2058 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 524683 T2058 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 524687 T2058 oass.SolrIndexSearcher.<init> Opening Searcher@b0c569 main
[junit4:junit4] 2> 524687 T2058 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 524688 T2058 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 524692 T2059 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b0c569 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 524693 T2058 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 524694 T2058 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42500 collection:control_collection shard:shard1
[junit4:junit4] 2> 524694 T2058 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 524699 T2058 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 524700 T2058 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 524700 T2058 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 524701 T2058 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42500/collection1/
[junit4:junit4] 2> 524701 T2058 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 524701 T2058 oasc.SyncStrategy.syncToMe http://127.0.0.1:42500/collection1/ has no replicas
[junit4:junit4] 2> 524701 T2058 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42500/collection1/
[junit4:junit4] 2> 524701 T2058 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 524986 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 524993 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 525006 T2058 oasc.ZkController.register We are http://127.0.0.1:42500/collection1/ and leader is http://127.0.0.1:42500/collection1/
[junit4:junit4] 2> 525006 T2058 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42500
[junit4:junit4] 2> 525006 T2058 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 525007 T2058 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 525007 T2058 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 525008 T2058 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 525009 T2033 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 525010 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 525011 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 525014 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 525016 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 525017 T2062 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9cf50 name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 525017 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 525019 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 525034 T2033 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 525128 T2033 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 525130 T2033 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37650
[junit4:junit4] 2> 525130 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 525131 T2033 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 525131 T2033 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950
[junit4:junit4] 2> 525131 T2033 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/solr.xml
[junit4:junit4] 2> 525132 T2033 oasc.CoreContainer.<init> New CoreContainer 32157864
[junit4:junit4] 2> 525132 T2033 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/'
[junit4:junit4] 2> 525133 T2033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/'
[junit4:junit4] 2> 525237 T2033 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 525237 T2033 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 525238 T2033 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 525238 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 525239 T2033 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 525239 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 525240 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 525240 T2033 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 525241 T2033 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 525241 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 525245 T2033 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 525246 T2033 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51660/solr
[junit4:junit4] 2> 525246 T2033 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 525247 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 525248 T2073 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a347 name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 525249 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 525251 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 525257 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 525258 T2075 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1954e32 name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 525259 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 525262 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 526264 T2033 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37650_
[junit4:junit4] 2> 526266 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37650_
[junit4:junit4] 2> 526270 T2062 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 526270 T2055 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 526270 T2075 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 526270 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 526277 T2076 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 526278 T2076 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 526497 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 526498 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42500_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42500"}
[junit4:junit4] 2> 526501 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37650_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37650"}
[junit4:junit4] 2> 526501 T2056 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 526501 T2056 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 526506 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 526507 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 526507 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 527279 T2076 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/collection1
[junit4:junit4] 2> 527279 T2076 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 527280 T2076 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 527280 T2076 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 527282 T2076 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/collection1/'
[junit4:junit4] 2> 527283 T2076 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/collection1/lib/README' to classloader
[junit4:junit4] 2> 527283 T2076 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 527331 T2076 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 527387 T2076 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 527388 T2076 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 527393 T2076 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 527870 T2076 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 527880 T2076 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 527892 T2076 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 527918 T2076 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 527928 T2076 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 527941 T2076 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 527942 T2076 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 527943 T2076 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 527943 T2076 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 527944 T2076 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 527944 T2076 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 527944 T2076 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 527945 T2076 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371680917950/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/
[junit4:junit4] 2> 527945 T2076 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a0b53e
[junit4:junit4] 2> 527946 T2076 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 527946 T2076 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1
[junit4:junit4] 2> 527947 T2076 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index/
[junit4:junit4] 2> 527947 T2076 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 527947 T2076 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index
[junit4:junit4] 2> 527951 T2076 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 527951 T2076 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 527954 T2076 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 527954 T2076 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 527955 T2076 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 527956 T2076 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 527956 T2076 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 527956 T2076 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 527957 T2076 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 527958 T2076 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 527958 T2076 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 527960 T2076 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 527964 T2076 oass.SolrIndexSearcher.<init> Opening Searcher@af53ed main
[junit4:junit4] 2> 527965 T2076 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 527965 T2076 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 527969 T2077 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@af53ed main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 527971 T2076 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 527972 T2076 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37650 collection:collection1 shard:shard1
[junit4:junit4] 2> 527972 T2076 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 527979 T2076 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 527981 T2076 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 527981 T2076 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 527981 T2076 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 527982 T2076 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 527982 T2076 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/ has no replicas
[junit4:junit4] 2> 527982 T2076 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 527982 T2076 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 528012 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 528018 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 528018 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 528019 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 528040 T2076 oasc.ZkController.register We are http://127.0.0.1:37650/collection1/ and leader is http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 528040 T2076 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37650
[junit4:junit4] 2> 528040 T2076 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 528041 T2076 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 528041 T2076 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 528043 T2076 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 528044 T2033 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 528044 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 528045 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 528142 T2033 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 528144 T2033 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49017
[junit4:junit4] 2> 528145 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 528145 T2033 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 528146 T2033 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963
[junit4:junit4] 2> 528146 T2033 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/solr.xml
[junit4:junit4] 2> 528146 T2033 oasc.CoreContainer.<init> New CoreContainer 31904430
[junit4:junit4] 2> 528147 T2033 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/'
[junit4:junit4] 2> 528147 T2033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/'
[junit4:junit4] 2> 528210 T2033 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 528211 T2033 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 528212 T2033 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 528212 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 528213 T2033 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 528213 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 528214 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 528214 T2033 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 528215 T2033 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 528215 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 528219 T2033 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 528220 T2033 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51660/solr
[junit4:junit4] 2> 528221 T2033 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 528226 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 528227 T2089 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e328f8 name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 528227 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 528229 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 528234 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 528242 T2091 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eeb800 name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 528242 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 528246 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 529249 T2033 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49017_
[junit4:junit4] 2> 529250 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49017_
[junit4:junit4] 2> 529252 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 529253 T2055 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 529253 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 529253 T2091 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 529253 T2075 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 529254 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 529254 T2062 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 529257 T2092 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 529258 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 529523 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 529523 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37650_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37650"}
[junit4:junit4] 2> 529527 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49017"}
[junit4:junit4] 2> 529527 T2056 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 529527 T2056 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 529536 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 529536 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 529536 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 529536 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 530259 T2092 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/collection1
[junit4:junit4] 2> 530259 T2092 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 530260 T2092 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 530260 T2092 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 530261 T2092 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/collection1/'
[junit4:junit4] 2> 530262 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/collection1/lib/README' to classloader
[junit4:junit4] 2> 530262 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 530296 T2092 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 530329 T2092 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 530331 T2092 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 530334 T2092 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 530689 T2092 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 530695 T2092 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 530697 T2092 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 530707 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 530710 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 530713 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 530713 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 530714 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 530714 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 530715 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 530715 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 530715 T2092 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 530716 T2092 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371680920963/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/
[junit4:junit4] 2> 530716 T2092 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a0b53e
[junit4:junit4] 2> 530716 T2092 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 530717 T2092 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2
[junit4:junit4] 2> 530717 T2092 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index/
[junit4:junit4] 2> 530717 T2092 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 530717 T2092 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index
[junit4:junit4] 2> 530845 T2092 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 530845 T2092 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 530847 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 530847 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 530847 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 530848 T2092 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 530848 T2092 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 530849 T2092 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 530849 T2092 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 530849 T2092 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 530850 T2092 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 530851 T2092 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 530854 T2092 oass.SolrIndexSearcher.<init> Opening Searcher@1c50c6a main
[junit4:junit4] 2> 530854 T2092 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 530855 T2092 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 530859 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c50c6a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 530861 T2092 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 530861 T2092 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49017 collection:collection1 shard:shard1
[junit4:junit4] 2> 530863 T2092 oasc.ZkController.register We are http://127.0.0.1:49017/collection1/ and leader is http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 530863 T2092 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49017
[junit4:junit4] 2> 530863 T2092 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 530864 T2092 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1849 name=collection1 org.apache.solr.core.SolrCore@3331b9 url=http://127.0.0.1:49017/collection1 node=127.0.0.1:49017_ C1849_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49017_, base_url=http://127.0.0.1:49017}
[junit4:junit4] 2> 530864 T2094 C1849 P49017 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 530864 T2092 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 530864 T2094 C1849 P49017 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 530864 T2094 C1849 P49017 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 530865 T2094 C1849 P49017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 530865 T2033 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 530865 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 530865 T2094 C1849 P49017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 530866 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 530869 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 530969 T2033 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 530972 T2033 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60040
[junit4:junit4] 2> 530973 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 530973 T2033 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 530974 T2033 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784
[junit4:junit4] 2> 530974 T2033 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/solr.xml
[junit4:junit4] 2> 530975 T2033 oasc.CoreContainer.<init> New CoreContainer 26154610
[junit4:junit4] 2> 530975 T2033 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/'
[junit4:junit4] 2> 530976 T2033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/'
[junit4:junit4] 2> 531030 T2033 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 531030 T2033 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 531030 T2033 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 531031 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 531031 T2033 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 531031 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 531032 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 531032 T2033 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 531032 T2033 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 531033 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 531035 T2033 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 531036 T2033 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51660/solr
[junit4:junit4] 2> 531036 T2033 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 531037 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 531039 T2106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56a43f name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 531039 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 531041 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 531043 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 531045 T2108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc266c name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 531045 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 531049 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 531056 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 531057 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49017"}
[junit4:junit4] 2> 531060 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 531060 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 531060 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 531060 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 531060 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 531870 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 531870 T2071 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:49017_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 532051 T2033 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60040_
[junit4:junit4] 2> 532052 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60040_
[junit4:junit4] 2> 532054 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 532054 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 532062 T2108 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 532062 T2055 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 532063 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532062 T2075 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 532062 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532063 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532064 T2091 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 532064 T2062 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 532069 T2109 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 532069 T2109 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 532565 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 532565 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60040_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60040"}
[junit4:junit4] 2> 532566 T2056 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 532566 T2056 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 532569 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532569 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532569 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532569 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 532569 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 533070 T2109 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/collection1
[junit4:junit4] 2> 533071 T2109 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 533071 T2109 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 533071 T2109 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 533072 T2109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/collection1/'
[junit4:junit4] 2> 533073 T2109 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/collection1/lib/README' to classloader
[junit4:junit4] 2> 533073 T2109 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 533102 T2109 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 533137 T2109 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 533139 T2109 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 533144 T2109 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 533542 T2109 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 533548 T2109 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 533551 T2109 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 533561 T2109 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 533564 T2109 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 533567 T2109 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 533568 T2109 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 533568 T2109 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 533568 T2109 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 533569 T2109 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 533569 T2109 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 533569 T2109 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 533570 T2109 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371680923784/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/
[junit4:junit4] 2> 533570 T2109 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a0b53e
[junit4:junit4] 2> 533570 T2109 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 533571 T2109 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3
[junit4:junit4] 2> 533571 T2109 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index/
[junit4:junit4] 2> 533571 T2109 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 533571 T2109 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index
[junit4:junit4] 2> 533574 T2109 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 533574 T2109 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 533576 T2109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 533576 T2109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 533577 T2109 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 533577 T2109 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 533578 T2109 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 533578 T2109 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 533578 T2109 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 533579 T2109 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 533579 T2109 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 533581 T2109 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 533583 T2109 oass.SolrIndexSearcher.<init> Opening Searcher@198cc05 main
[junit4:junit4] 2> 533584 T2109 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 533584 T2109 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 533589 T2110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@198cc05 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 533591 T2109 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 533592 T2109 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60040 collection:collection1 shard:shard1
[junit4:junit4] 2> 533594 T2109 oasc.ZkController.register We are http://127.0.0.1:60040/collection1/ and leader is http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 533594 T2109 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60040
[junit4:junit4] 2> 533594 T2109 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 533594 T2109 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1850 name=collection1 org.apache.solr.core.SolrCore@111c55 url=http://127.0.0.1:60040/collection1 node=127.0.0.1:60040_ C1850_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:60040_, base_url=http://127.0.0.1:60040}
[junit4:junit4] 2> 533595 T2111 C1850 P60040 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 533595 T2109 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 533595 T2111 C1850 P60040 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 533596 T2111 C1850 P60040 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 533596 T2111 C1850 P60040 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 533596 T2033 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 533597 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 533597 T2111 C1850 P60040 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 533597 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 533602 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 533665 T2033 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 533668 T2033 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51821
[junit4:junit4] 2> 533668 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 533669 T2033 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 533669 T2033 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516
[junit4:junit4] 2> 533669 T2033 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/solr.xml
[junit4:junit4] 2> 533669 T2033 oasc.CoreContainer.<init> New CoreContainer 27458503
[junit4:junit4] 2> 533670 T2033 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/'
[junit4:junit4] 2> 533670 T2033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/'
[junit4:junit4] 2> 533728 T2033 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 533729 T2033 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 533729 T2033 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 533730 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 533730 T2033 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 533731 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 533731 T2033 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 533731 T2033 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 533732 T2033 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 533732 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 533735 T2033 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 533735 T2033 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51660/solr
[junit4:junit4] 2> 533736 T2033 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 533736 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 533738 T2123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e6d0e2 name:ZooKeeperConnection Watcher:127.0.0.1:51660 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 533739 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 533741 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 533746 T2033 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 533747 T2125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@149707d name:ZooKeeperConnection Watcher:127.0.0.1:51660/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 533747 T2033 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 533751 T2033 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1851 name=collection1 org.apache.solr.core.SolrCore@3331b9 url=http://127.0.0.1:49017/collection1 node=127.0.0.1:49017_ C1851_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49017_, base_url=http://127.0.0.1:49017}
[junit4:junit4] 2> 533871 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37650/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 533871 T2094 C1851 P49017 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49017 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 533871 T2094 C1851 P49017 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 533872 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 533872 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 533872 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 533873 T2094 C1851 P49017 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 533873 T2094 C1851 P49017 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37650/collection1/. core=collection1
[junit4:junit4] 2> 533873 T2094 C1851 P49017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1852 name=collection1 org.apache.solr.core.SolrCore@1343c2a url=http://127.0.0.1:37650/collection1 node=127.0.0.1:37650_ C1852_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37650_, base_url=http://127.0.0.1:37650, leader=true}
[junit4:junit4] 2> 533880 T2068 C1852 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 533883 T2070 C1852 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 533885 T2070 C1852 P37650 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 533886 T2070 C1852 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 533893 T2070 C1852 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 533894 T2070 C1852 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 533895 T2070 C1852 P37650 oass.SolrIndexSearcher.<init> Opening Searcher@14b5b53 realtime
[junit4:junit4] 2> 533896 T2070 C1852 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 533896 T2070 C1852 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 533897 T2094 C1851 P49017 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 533898 T2094 C1851 P49017 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 533899 T2069 C1852 P37650 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 533900 T2069 C1852 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 533900 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 533900 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 533901 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 533903 T2068 C1852 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 533904 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 533905 T2094 C1851 P49017 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819
[junit4:junit4] 2> 533905 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819 lockFactory=org.apache.lucene.store.NativeFSLockFactory@420cc0 fullCopy=false
[junit4:junit4] 2> 533908 T2070 C1852 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 533910 T2094 C1851 P49017 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 533914 T2094 C1851 P49017 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 533914 T2094 C1851 P49017 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 533916 T2094 C1851 P49017 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 533917 T2094 C1851 P49017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 533917 T2094 C1851 P49017 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 533918 T2094 C1851 P49017 oass.SolrIndexSearcher.<init> Opening Searcher@113b61d main
[junit4:junit4] 2> 533919 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@113b61d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 533919 T2094 C1851 P49017 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819;done=true>>]
[junit4:junit4] 2> 533920 T2094 C1851 P49017 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819
[junit4:junit4] 2> 533920 T2094 C1851 P49017 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index.20130620082846819
[junit4:junit4] 2> 533920 T2094 C1851 P49017 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 533921 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 533921 T2094 C1851 P49017 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 533921 T2094 C1851 P49017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 533923 T2094 C1851 P49017 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 534075 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 534076 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60040_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60040"}
[junit4:junit4] 2> 534079 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49017"}
[junit4:junit4] 2> 534094 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534094 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534094 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534094 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534094 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534094 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534602 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 534603 T2071 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:60040_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 534754 T2033 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51821_
[junit4:junit4] 2> 534755 T2033 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51821_
[junit4:junit4] 2> 534757 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534757 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534757 T2055 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534757 T2125 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534758 T2108 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534758 T2075 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534758 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534757 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534758 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534758 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534758 T2091 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534759 T2062 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 534764 T2128 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 534765 T2128 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 535607 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 535608 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51821_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51821"}
[junit4:junit4] 2> 535608 T2056 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 535608 T2056 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 535611 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535611 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535611 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535611 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535611 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535611 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535766 T2128 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/collection1
[junit4:junit4] 2> 535766 T2128 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 535767 T2128 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 535767 T2128 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 535768 T2128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/collection1/'
[junit4:junit4] 2> 535768 T2128 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/collection1/lib/README' to classloader
[junit4:junit4] 2> 535769 T2128 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 535809 T2128 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 535853 T2128 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 535854 T2128 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 535857 T2128 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 536260 T2128 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 536266 T2128 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 536269 T2128 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 536278 T2128 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 536281 T2128 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 536284 T2128 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 536285 T2128 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 536286 T2128 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 536286 T2128 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 536287 T2128 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 536287 T2128 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 536287 T2128 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 536287 T2128 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371680926516/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/
[junit4:junit4] 2> 536287 T2128 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a0b53e
[junit4:junit4] 2> 536288 T2128 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 536288 T2128 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4
[junit4:junit4] 2> 536288 T2128 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index/
[junit4:junit4] 2> 536289 T2128 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 536289 T2128 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index
[junit4:junit4] 2> 536292 T2128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 536292 T2128 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 536294 T2128 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 536294 T2128 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 536294 T2128 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 536295 T2128 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 536295 T2128 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 536296 T2128 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 536296 T2128 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 536296 T2128 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 536297 T2128 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 536299 T2128 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 536301 T2128 oass.SolrIndexSearcher.<init> Opening Searcher@1dcd510 main
[junit4:junit4] 2> 536302 T2128 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 536302 T2128 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 536307 T2129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dcd510 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 536308 T2128 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 536309 T2128 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51821 collection:collection1 shard:shard1
[junit4:junit4] 2> 536311 T2128 oasc.ZkController.register We are http://127.0.0.1:51821/collection1/ and leader is http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 536311 T2128 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51821
[junit4:junit4] 2> 536311 T2128 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 536311 T2128 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1853 name=collection1 org.apache.solr.core.SolrCore@139d291 url=http://127.0.0.1:51821/collection1 node=127.0.0.1:51821_ C1853_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51821_, base_url=http://127.0.0.1:51821}
[junit4:junit4] 2> 536312 T2130 C1853 P51821 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 536312 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 536312 T2128 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 536312 T2130 C1853 P51821 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 536312 T2130 C1853 P51821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 536313 T2033 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 536313 T2033 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 536313 T2130 C1853 P51821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 536313 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 536317 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 536318 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 536318 T2033 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 536318 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 536319 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1854 name=collection1 org.apache.solr.core.SolrCore@111c55 url=http://127.0.0.1:60040/collection1 node=127.0.0.1:60040_ C1854_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:60040_, base_url=http://127.0.0.1:60040}
[junit4:junit4] 2> 536604 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37650/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 536604 T2111 C1854 P60040 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60040 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 536604 T2111 C1854 P60040 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 536604 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 536604 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 536605 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 536605 T2111 C1854 P60040 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 536605 T2111 C1854 P60040 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37650/collection1/. core=collection1
[junit4:junit4] 2> 536605 T2111 C1854 P60040 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1855 name=collection1 org.apache.solr.core.SolrCore@1343c2a url=http://127.0.0.1:37650/collection1 node=127.0.0.1:37650_ C1855_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37650_, base_url=http://127.0.0.1:37650, leader=true}
[junit4:junit4] 2> 536606 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 536608 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 536613 T2069 C1855 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 536614 T2069 C1855 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 536614 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 536614 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 536615 T2111 C1854 P60040 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 536615 T2111 C1854 P60040 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 536617 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 536618 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 536618 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 536618 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 536620 T2069 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 536620 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 536621 T2111 C1854 P60040 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535
[junit4:junit4] 2> 536621 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535 lockFactory=org.apache.lucene.store.NativeFSLockFactory@72f4f7 fullCopy=false
[junit4:junit4] 2> 536623 T2068 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 536624 T2111 C1854 P60040 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 536626 T2111 C1854 P60040 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 536626 T2111 C1854 P60040 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 536628 T2111 C1854 P60040 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 536628 T2111 C1854 P60040 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 536628 T2111 C1854 P60040 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 536629 T2111 C1854 P60040 oass.SolrIndexSearcher.<init> Opening Searcher@76caad main
[junit4:junit4] 2> 536629 T2110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76caad main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 536630 T2111 C1854 P60040 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535;done=true>>]
[junit4:junit4] 2> 536630 T2111 C1854 P60040 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535
[junit4:junit4] 2> 536630 T2111 C1854 P60040 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index.20130620082849535
[junit4:junit4] 2> 536630 T2111 C1854 P60040 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 536631 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 536631 T2111 C1854 P60040 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 536631 T2111 C1854 P60040 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 536643 T2111 C1854 P60040 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 537114 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 537115 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51821_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51821"}
[junit4:junit4] 2> 537118 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60040_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60040"}
[junit4:junit4] 2> 537121 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537121 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537121 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537121 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537121 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537121 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 537318 T2071 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 537319 T2071 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:51821_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 537320 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 538321 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1853_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51821_, base_url=http://127.0.0.1:51821}
[junit4:junit4] 2> 539319 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37650/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 539320 T2130 C1853 P51821 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51821 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 539320 T2130 C1853 P51821 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 539320 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 539320 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 539320 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 539321 T2130 C1853 P51821 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 539321 T2130 C1853 P51821 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37650/collection1/. core=collection1
[junit4:junit4] 2> 539321 T2130 C1853 P51821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 539322 T2071 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 539323 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 539326 T2068 C1855 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 539333 T2068 C1855 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 539334 T2068 C1855 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 539335 T2068 C1855 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 539335 T2068 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 539336 T2130 C1853 P51821 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 539336 T2130 C1853 P51821 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 539338 T2071 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 539338 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 539339 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 539339 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 539341 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 539341 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 539342 T2130 C1853 P51821 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256
[junit4:junit4] 2> 539343 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b7090 fullCopy=false
[junit4:junit4] 2> 539345 T2068 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 539346 T2130 C1853 P51821 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 539348 T2130 C1853 P51821 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 539349 T2130 C1853 P51821 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 539350 T2130 C1853 P51821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 539351 T2130 C1853 P51821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 539351 T2130 C1853 P51821 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 539352 T2130 C1853 P51821 oass.SolrIndexSearcher.<init> Opening Searcher@1f23a00 main
[junit4:junit4] 2> 539353 T2129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f23a00 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 539353 T2130 C1853 P51821 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256;done=true>>]
[junit4:junit4] 2> 539353 T2130 C1853 P51821 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256
[junit4:junit4] 2> 539354 T2130 C1853 P51821 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index.20130620082852256
[junit4:junit4] 2> 539354 T2130 C1853 P51821 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 539354 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 539354 T2130 C1853 P51821 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 539355 T2130 C1853 P51821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 539356 T2130 C1853 P51821 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 540127 T2056 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 540128 T2056 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51821_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51821"}
[junit4:junit4] 2> 540145 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540146 T2075 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540146 T2055 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540145 T2062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540145 T2091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540145 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 540324 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 540325 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1856 name=collection1 org.apache.solr.core.SolrCore@19be706 url=http://127.0.0.1:42500/collection1 node=127.0.0.1:42500_ C1856_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:42500_, base_url=http://127.0.0.1:42500, leader=true}
[junit4:junit4] 2> 540331 T2046 C1856 P42500 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 540333 T2046 C1856 P42500 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 540333 T2046 C1856 P42500 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 540343 T2046 C1856 P42500 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 540344 T2046 C1856 P42500 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 540345 T2046 C1856 P42500 oass.SolrIndexSearcher.<init> Opening Searcher@5a66b2 main
[junit4:junit4] 2> 540345 T2046 C1856 P42500 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 540346 T2059 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a66b2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 540346 T2046 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> ASYNC NEW_CORE C1857 name=collection1 org.apache.solr.core.SolrCore@3331b9 url=http://127.0.0.1:49017/collection1 node=127.0.0.1:49017_ C1857_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49017_, base_url=http://127.0.0.1:49017}
[junit4:junit4] 2> 540353 T2087 C1857 P49017 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:37650/collection1/, StdNode: http://127.0.0.1:49017/collection1/, StdNode: http://127.0.0.1:60040/collection1/, StdNode: http://127.0.0.1:51821/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 540356 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 540356 T2085 C1857 P49017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1854_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60040_, base_url=http://127.0.0.1:60040}
[junit4:junit4] 2> 540360 T2101 C1854 P60040 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1853_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51821_, base_url=http://127.0.0.1:51821}
[junit4:junit4] 2> 540362 T2121 C1853 P51821 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 540373 T2069 C1855 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 540374 T2069 C1855 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 540374 T2069 C1855 P37650 oass.SolrIndexSearcher.<init> Opening Searcher@14f5f40 main
[junit4:junit4] 2> 540375 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 540375 T2077 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14f5f40 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 540376 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 540393 T2101 C1854 P60040 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 540394 T2101 C1854 P60040 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 540393 T2085 C1857 P49017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 540395 T2085 C1857 P49017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 540395 T2101 C1854 P60040 oass.SolrIndexSearcher.<init> Opening Searcher@57023f main
[junit4:junit4] 2> 540395 T2085 C1857 P49017 oass.SolrIndexSearcher.<init> Opening Searcher@5a758 main
[junit4:junit4] 2> 540395 T2101 C1854 P60040 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 540396 T2085 C1857 P49017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 540396 T2110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57023f main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 540396 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a758 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 540396 T2101 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> 540397 T2085 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 41
[junit4:junit4] 2> 540403 T2121 C1853 P51821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 540404 T2121 C1853 P51821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 540404 T2121 C1853 P51821 oass.SolrIndexSearcher.<init> Opening Searcher@12b5883 main
[junit4:junit4] 2> 540405 T2121 C1853 P51821 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 540405 T2129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12b5883 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 540406 T2121 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 44
[junit4:junit4] 2> 540406 T2087 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 53
[junit4:junit4] 2> 540407 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 540409 T2071 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 540410 T2086 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 540411 T2104 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 540412 T2118 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 542416 T2051 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438311708444590080)} 0 2
[junit4:junit4] 2> 542426 T2084 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438311708449832960&update.from=http://127.0.0.1:37650/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438311708449832960)} 0 1
[junit4:junit4] 2> 542427 T2103 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438311708449832960&update.from=http://127.0.0.1:37650/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438311708449832960)} 0 1
[junit4:junit4] 2> 542427 T2121 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438311708449832960&update.from=http://127.0.0.1:37650/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438311708449832960)} 0 1
[junit4:junit4] 2> 542428 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438311708449832960)} 0 8
[junit4:junit4] 2> 542428 T2086 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 10
[junit4:junit4] 2> 542432 T2046 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438311708460318720)]} 0 1
[junit4:junit4] 2> 542439 T2101 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438311708464513024)]} 0 2
[junit4:junit4] 2> 542439 T2087 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438311708464513024)]} 0 2
[junit4:junit4] 2> 542440 T2119 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438311708464513024)]} 0 3
[junit4:junit4] 2> 542441 T2071 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60040/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438311708464513024)]} 0 7
[junit4:junit4] 2> 542441 T2102 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 8
[junit4:junit4] 2> 542444 T2049 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438311708473950208)]} 0 1
[junit4:junit4] 2> 542448 T2103 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438311708476047360)]} 0 1
[junit4:junit4] 2> 542448 T2085 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438311708476047360)]} 0 1
[junit4:junit4] 2> 542449 T2120 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438311708476047360)]} 0 1
[junit4:junit4] 2> 542450 T2068 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438311708476047360)]} 0 5
[junit4:junit4] 2> 542452 T2050 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438311708483387392)]} 0 0
[junit4:junit4] 2> 542456 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49017/collection1/]
[junit4:junit4] 2> 542457 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49017/collection1/ against:[http://127.0.0.1:49017/collection1/] result:true
[junit4:junit4] 2> 542457 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:60040/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 542457 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51821/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 542461 T2118 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[2 (1438311708489678848)]} 0 1
[junit4:junit4] 2> 542461 T2104 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[2 (1438311708489678848)]} 0 1
[junit4:junit4] 2> 542462 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49017/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[2 (1438311708489678848)]} 0 6
[junit4:junit4] 2> 542462 T2084 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 542465 T2051 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438311708495970304)]} 0 1
[junit4:junit4] 2> 542467 T2071 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49017/collection1/]
[junit4:junit4] 2> 542468 T2071 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49017/collection1/ against:[http://127.0.0.1:49017/collection1/] result:true
[junit4:junit4] 2> 542468 T2071 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:60040/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 542468 T2071 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51821/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 542471 T2101 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[3 (1438311708501213184)]} 0 1
[junit4:junit4] 2> 542471 T2119 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[3 (1438311708501213184)]} 0 1
[junit4:junit4] 2> 542472 T2071 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51821/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[3 (1438311708501213184)]} 0 5
[junit4:junit4] 2> 542472 T2121 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&test.distrib.skip.servers=http://127.0.0.1:60040/collection1/&wt=javabin&version=2} {add=[3]} 0 6
[junit4:junit4] 2> 542474 T2046 C1856 P42500 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542513 T2046 C1856 P42500 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 542514 T2046 C1856 P42500 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 542515 T2046 C1856 P42500 oass.SolrIndexSearcher.<init> Opening Searcher@2170b2 main
[junit4:junit4] 2> 542516 T2046 C1856 P42500 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542516 T2059 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2170b2 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542517 T2046 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 542519 T2070 C1855 P37650 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:37650/collection1/, StdNode: http://127.0.0.1:49017/collection1/, StdNode: http://127.0.0.1:60040/collection1/, StdNode: http://127.0.0.1:51821/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 542520 T2068 C1855 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542521 T2086 C1857 P49017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542521 T2120 C1853 P51821 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542521 T2102 C1854 P60040 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542574 T2102 C1854 P60040 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 542574 T2102 C1854 P60040 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 542574 T2086 C1857 P49017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 542575 T2086 C1857 P49017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 542574 T2120 C1853 P51821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 542574 T2068 C1855 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 542576 T2102 C1854 P60040 oass.SolrIndexSearcher.<init> Opening Searcher@1d1e5bb main
[junit4:junit4] 2> 542576 T2068 C1855 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 542576 T2120 C1853 P51821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 542577 T2102 C1854 P60040 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542576 T2086 C1857 P49017 oass.SolrIndexSearcher.<init> Opening Searcher@b3fc2e main
[junit4:junit4] 2> 542578 T2068 C1855 P37650 oass.SolrIndexSearcher.<init> Opening Searcher@1485f86 main
[junit4:junit4] 2> 542578 T2086 C1857 P49017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542577 T2110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d1e5bb main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542579 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b3fc2e main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):c2)}
[junit4:junit4] 2> 542579 T2077 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:302)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2>
[junit4:junit4] 2> 542578 T2068 C1855 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542578 T2120 C1853 P51821 oass.SolrIndexSearcher.<init> Opening Searcher@10b218c main
[junit4:junit4] 2> 542579 T2077 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1485f86 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542580 T2120 C1853 P51821 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542579 T2086 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 542581 T2129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10b218c main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542579 T2102 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 542581 T2120 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 60
[junit4:junit4] 2> 542580 T2068 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 60
[junit4:junit4] 2> 542582 T2070 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 542582 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 542583 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 542584 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 542586 T2069 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 542588 T2087 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=1
[junit4:junit4] 2> 542590 T2103 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 542592 T2118 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 542592 T2033 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 542596 T2104 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 542596 T2104 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 542599 T2071 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 542599 T2071 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37650/collection1/
[junit4:junit4] 2> 542599 T2071 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37650 START replicas=[http://127.0.0.1:49017/collection1/, http://127.0.0.1:60040/collection1/, http://127.0.0.1:51821/collection1/] nUpdates=100
[junit4:junit4] 2> 542601 T2121 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542601 T2084 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542601 T2101 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542601 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Received 5 versions from 127.0.0.1:51821/collection1/
[junit4:junit4] 2> 542602 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Our versions are newer. ourLowThreshold=1438311708449832960 otherHigh=1438311708489678848
[junit4:junit4] 2> 542602 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Received 3 versions from 127.0.0.1:49017/collection1/
[junit4:junit4] 2> 542603 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Our versions are newer. ourLowThreshold=1438311708449832960 otherHigh=1438311708476047360
[junit4:junit4] 2> 542603 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Received 5 versions from 127.0.0.1:60040/collection1/
[junit4:junit4] 2> 542603 T2071 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37650 Our versions are newer. ourLowThreshold=1438311708449832960 otherHigh=1438311708489678848
[junit4:junit4] 2> 542603 T2071 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37650 DONE. sync succeeded
[junit4:junit4] 2> 542604 T2071 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 542604 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: try and ask http://127.0.0.1:49017/collection1/ to sync
[junit4:junit4] 2> 542604 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: try and ask http://127.0.0.1:60040/collection1/ to sync
[junit4:junit4] 2> 542605 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: try and ask http://127.0.0.1:51821/collection1/ to sync
[junit4:junit4] 2> 542605 T2086 C1857 P49017 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49017 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 542605 T2102 C1854 P60040 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60040 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 542606 T2120 C1853 P51821 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51821 START replicas=[http://127.0.0.1:37650/collection1/] nUpdates=100
[junit4:junit4] 2> 542607 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542607 T2069 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542607 T2068 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542608 T2102 C1854 P60040 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60040 Received 5 versions from 127.0.0.1:37650/collection1/
[junit4:junit4] 2> 542608 T2086 C1857 P49017 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:49017 Received 5 versions from 127.0.0.1:37650/collection1/
[junit4:junit4] 2> 542608 T2120 C1853 P51821 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51821 Received 5 versions from 127.0.0.1:37650/collection1/
[junit4:junit4] 2> 542609 T2086 C1857 P49017 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:49017 Requesting updates from 127.0.0.1:37650/collection1/n=2 versions=[1438311708501213184, 1438311708489678848]
[junit4:junit4] 2> 542608 T2102 C1854 P60040 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60040 Our versions are newer. ourLowThreshold=1438311708449832960 otherHigh=1438311708489678848
[junit4:junit4] 2> 542609 T2120 C1853 P51821 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51821 Our versions are newer. ourLowThreshold=1438311708449832960 otherHigh=1438311708489678848
[junit4:junit4] 2> 542610 T2120 C1853 P51821 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51821 DONE. sync succeeded
[junit4:junit4] 2> 542610 T2102 C1854 P60040 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60040 DONE. sync succeeded
[junit4:junit4] 2> 542610 T2120 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:37650/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 542611 T2102 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:37650/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=6
[junit4:junit4] 2> 542611 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: sync completed with http://127.0.0.1:51821/collection1/
[junit4:junit4] 2> 542611 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: sync completed with http://127.0.0.1:60040/collection1/
[junit4:junit4] 2> 542611 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438311708501213184,1438311708489678848&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 542615 T2086 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438311708489678848), 3 (1438311708501213184)]} 0 3
[junit4:junit4] 2> 542616 T2086 C1857 P49017 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49017 DONE. sync succeeded
[junit4:junit4] 2> 542616 T2086 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:37650/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=11
[junit4:junit4] 2> 542616 T2071 oasc.SyncStrategy.syncToMe http://127.0.0.1:37650/collection1/: sync completed with http://127.0.0.1:49017/collection1/
[junit4:junit4] 2> 542617 T2071 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=18
[junit4:junit4] 2> 542617 T2104 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=21
[junit4:junit4] 2> 542618 T2033 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 542618 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 542619 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 542619 T2033 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 542621 T2050 C1856 P42500 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542663 T2050 C1856 P42500 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 542664 T2050 C1856 P42500 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 542665 T2050 C1856 P42500 oass.SolrIndexSearcher.<init> Opening Searcher@18b1a7 main
[junit4:junit4] 2> 542665 T2050 C1856 P42500 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542666 T2059 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18b1a7 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542666 T2050 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 45
[junit4:junit4] 2> 542668 T2087 C1857 P49017 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:37650/collection1/, StdNode: http://127.0.0.1:49017/collection1/, StdNode: http://127.0.0.1:60040/collection1/, StdNode: http://127.0.0.1:51821/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 542670 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542670 T2085 C1857 P49017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542670 T2103 C1854 P60040 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542670 T2118 C1853 P51821 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 542683 T2069 C1855 P37650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 542684 T2069 C1855 P37650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 542685 T2069 C1855 P37650 oass.SolrIndexSearcher.<init> Opening Searcher@fb2f7 main
[junit4:junit4] 2> 542685 T2069 C1855 P37650 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542686 T2077 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@fb2f7 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542686 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> 542703 T2103 C1854 P60040 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 542704 T2103 C1854 P60040 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 542703 T2118 C1853 P51821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 542705 T2103 C1854 P60040 oass.SolrIndexSearcher.<init> Opening Searcher@2cd475 main
[junit4:junit4] 2> 542705 T2118 C1853 P51821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 542705 T2103 C1854 P60040 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542712 T2118 C1853 P51821 oass.SolrIndexSearcher.<init> Opening Searcher@1c3d460 main
[junit4:junit4] 2> 542712 T2110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cd475 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542713 T2118 C1853 P51821 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542713 T2103 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 542713 T2129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c3d460 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 542714 T2118 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 44
[junit4:junit4] 2> 542743 T2085 C1857 P49017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371680914174/jetty2/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 542744 T2085 C1857 P49017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 542746 T2085 C1857 P49017 oass.SolrIndexSearcher.<init> Opening Searcher@abcff3 main
[junit4:junit4] 2> 542748 T2085 C1857 P49017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 542749 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@abcff3 main{StandardDirectoryReader(segments_5:5:nrt _0(4.4):c2 _1(4.4):c2)}
[junit4:junit4] 2> 542750 T2085 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 80
[junit4:junit4] 2> 542751 T2087 C1857 P49017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 83
[junit4:junit4] 2> 542752 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 542754 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 542756 T2084 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 542758 T2101 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 542760 T2119 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 544762 T2033 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:2:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:37650_","base_url":"http://127.0.0.1:37650","leader":"true"}
[junit4:junit4] 2> 544764 T2070 C1855 P37650 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:3:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:49017_","base_url":"http://127.0.0.1:49017"}
[junit4:junit4] 2> 544766 T2084 C1857 P49017 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:4:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:60040_","base_url":"http://127.0.0.1:60040"}
[junit4:junit4] 2> 544768 T2101 C1854 P60040 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:5:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:51821_","base_url":"http://127.0.0.1:51821"}
[junit4:junit4] 2> 544771 T2118 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> 544773 T2119 C1853 P51821 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&wt=javabin&version=2} hits=4 status=0 QTime=0
[junit4:junit4] 2> 544778 T2049 C1856 P42500 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1438311710920278016)]} 0 2
[junit4:junit4] 2> 544781 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:49017/collection1/]
[junit4:junit4] 2> 544782 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:49017/collection1/ against:[http://127.0.0.1:49017/collection1/] result:true
[junit4:junit4] 2> 544782 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:60040/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 544782 T2069 C1855 P37650 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51821/collection1/ against:[http://127.0.0.1:49017/collection1/] result:false
[junit4:junit4] 2> 544788 T2104 C1854 P60040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[4 (1438311710926569472)]} 0 2
[junit4:junit4] 2> 544788 T2121 C1853 P51821 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37650/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[4 (1438311710926569472)]} 0 2
[junit4:junit4] 2> 544789 T2069 C1855 P37650 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60040/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:49017/collection1/&wt=javabin&version=2} {add=[4 (1438311710926569472)]} 0 8
[junit4:junit4] 2> 544789 T2
[...truncated too long message...]
7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:42500_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:42500",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:60040_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37650_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:51821_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:42500_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894480391569422-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894480391569420-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894480391569418-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:60040_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60040"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89894480391569411-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:42500_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:42500"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89894480391569420-127.0.0.1:51821_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894480391569422-127.0.0.1:37650_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894480391569418-127.0.0.1:60040_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894480391569411-127.0.0.1:42500_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89894480391569411-127.0.0.1:42500_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=EC67CF367991BAF3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl_BE -Dtests.timezone=Australia/Currie -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 101s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:37650/collection1lastClient and got 258 from http://127.0.0.1:60040/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([EC67CF367991BAF3:6D81412E0ECEDACF]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 622048 T2033 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 100802 T2032 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4] 2> 622178 T2108 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 622179 T2108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 622179 T2108 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 623255 T2125 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 623255 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 623255 T2125 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=199), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=199)), sim=DefaultSimilarity, locale=nl_BE, timezone=Australia/Currie
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Sun Microsystems Inc. 1.6.0_45 (32-bit)/cpus=8,threads=3,free=143206648,total=300548096
[junit4:junit4] 2> NOTE: All tests run in this JVM: [RequiredFieldsTest, ZkNodePropsTest, TestUpdate, CopyFieldTest, TestFieldTypeCollectionResource, TestUtils, TestManagedSchemaFieldResource, TestCSVResponseWriter, XsltUpdateRequestHandlerTest, BadIndexSchemaTest, ClusterStateTest, OpenCloseCoreStressTest, UpdateParamsTest, TestSuggestSpellingConverter, TestZkChroot, TestSolrQueryParserDefaultOperatorResource, DirectSolrSpellCheckerTest, RequestHandlersTest, HighlighterTest, SimplePostToolTest, TestCollationField, TestUniqueKeyFieldResource, TestAnalyzedSuggestions, TestOmitPositions, EchoParamsTest, FieldAnalysisRequestHandlerTest, HighlighterConfigTest, DistanceFunctionTest, SpellCheckComponentTest, TestSolrXml, TestCopyFieldCollectionResource, SolrIndexConfigTest, StatsComponentTest, JSONWriterTest, PingRequestHandlerTest, TestAddFieldRealTimeGet, TestStressLucene, IndexBasedSpellCheckerTest, TestIndexingPerformance, TestMergePolicyConfig, BasicDistributedZk2Test, DateMathParserTest, TestDistributedGrouping, SuggesterTSTTest, TestRTGBase, SuggesterWFSTTest, SolrCmdDistributorTest, CoreContainerCoreInitFailuresTest, IndexSchemaRuntimeFieldTest, SpellPossibilityIteratorTest, ChaosMonkeySafeLeaderTest, StandardRequestHandlerTest, TestHashPartitioner, HardAutoCommitTest, DocValuesMultiTest, TestRecovery, TestFieldTypeResource, TestFiltering, DebugComponentTest, NoCacheHeaderTest, TestDefaultSearchFieldResource, TestLMJelinekMercerSimilarityFactory, TestRangeQuery, TestJmxIntegration, NotRequiredUniqueKeyTest, FileUtilsTest, DisMaxRequestHandlerTest, TestMultiCoreConfBootstrap, TestDynamicFieldCollectionResource, PreAnalyzedUpdateProcessorTest, CollectionsAPIDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 102.09s, 1 test, 1 failure <<< FAILURES!
[...truncated 511 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1255 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 39 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.6.0_45 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux
(32bit/jrockit-jdk1.6.0_37-R28.2.5-4.1.0) - Build # 6145 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6145/
Java: 32bit/jrockit-jdk1.6.0_37-R28.2.5-4.1.0 -XnoOpt
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:51911/_mvo/hr/collection1lastClient and got 251 from http://127.0.0.1:37425/_mvo/hr/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51911/_mvo/hr/collection1lastClient and got 251 from http://127.0.0.1:37425/_mvo/hr/collection1
at __randomizedtesting.SeedInfo.seed([1106578B97D21F1D:90E0D993E08D7F21]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:774)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:683)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:44)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 8950 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 726398 T1900 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_mvo/hr
[junit4:junit4] 2> 726424 T1900 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371686733401
[junit4:junit4] 2> 726425 T1900 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 726425 T1901 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 726525 T1900 oasc.ZkTestServer.run start zk server on port:58777
[junit4:junit4] 2> 726527 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726676 T1907 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b067621 name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726676 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726677 T1900 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 726681 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726682 T1909 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ae7e998 name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726682 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726682 T1900 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 726684 T1900 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 726687 T1900 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 726688 T1900 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 726691 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 726691 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 726694 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 726695 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 726698 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 726699 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 726701 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 726701 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 726704 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 726704 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 726707 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 726707 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 726709 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 726710 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 726712 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 726713 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 726715 T1900 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 726716 T1900 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 726814 T1900 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 726817 T1900 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46509
[junit4:junit4] 2> 726818 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 726818 T1900 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 726820 T1900 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695
[junit4:junit4] 2> 726821 T1900 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/solr.xml
[junit4:junit4] 2> 726821 T1900 oasc.CoreContainer.<init> New CoreContainer 465031871
[junit4:junit4] 2> 726822 T1900 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/'
[junit4:junit4] 2> 726822 T1900 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/'
[junit4:junit4] 2> 726888 T1900 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 726889 T1900 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 726889 T1900 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 726889 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 726890 T1900 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 726890 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 726890 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 726891 T1900 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 726891 T1900 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 726891 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 726895 T1900 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 726896 T1900 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58777/solr
[junit4:junit4] 2> 726896 T1900 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 726897 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726898 T1920 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bdb5590 name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726898 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726899 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 726903 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 726904 T1922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bdc4c8c name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 726904 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 726905 T1900 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 726907 T1900 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 726910 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 726911 T1900 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46509__mvo%2Fhr
[junit4:junit4] 2> 726912 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46509__mvo%2Fhr
[junit4:junit4] 2> 726914 T1900 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 726918 T1900 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 726919 T1900 oasc.Overseer.start Overseer (id=89894861760299011-127.0.0.1:46509__mvo%2Fhr-n_0000000000) starting
[junit4:junit4] 2> 726921 T1900 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 726923 T1924 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 726926 T1900 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 726927 T1900 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 726928 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 726930 T1923 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 726933 T1925 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 726933 T1925 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 728433 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 728433 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:46509__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46509/_mvo/hr"}
[junit4:junit4] 2> 728434 T1923 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 728434 T1923 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 728437 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 728934 T1925 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/collection1
[junit4:junit4] 2> 728935 T1925 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 728935 T1925 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 728936 T1925 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 728937 T1925 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/collection1/'
[junit4:junit4] 2> 728937 T1925 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/collection1/lib/README' to classloader
[junit4:junit4] 2> 728938 T1925 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 728972 T1925 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 729088 T1925 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 729089 T1925 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 729098 T1925 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 729606 T1925 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 729613 T1925 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 729617 T1925 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 729627 T1925 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 729631 T1925 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 729634 T1925 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 729635 T1925 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 729636 T1925 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 729636 T1925 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 729637 T1925 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 729637 T1925 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 729637 T1925 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 729644 T1925 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371686733695/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/
[junit4:junit4] 2> 729644 T1925 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ae7c8a9
[junit4:junit4] 2> 729644 T1925 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 729645 T1925 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data
[junit4:junit4] 2> 729645 T1925 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index/
[junit4:junit4] 2> 729645 T1925 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 729646 T1925 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index
[junit4:junit4] 2> 729650 T1925 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 729650 T1925 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 729651 T1925 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 729651 T1925 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 729652 T1925 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 729652 T1925 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 729653 T1925 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 729653 T1925 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 729653 T1925 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 729653 T1925 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 729654 T1925 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 729656 T1925 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 729659 T1925 oass.SolrIndexSearcher.<init> Opening Searcher@1b808213 main
[junit4:junit4] 2> 729659 T1925 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 729660 T1925 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 729663 T1926 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b808213 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 729665 T1925 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 729665 T1925 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46509/_mvo/hr collection:control_collection shard:shard1
[junit4:junit4] 2> 729665 T1925 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 729670 T1925 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 729671 T1925 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 729671 T1925 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 729671 T1925 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46509/_mvo/hr/collection1/
[junit4:junit4] 2> 729672 T1925 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 729672 T1925 oasc.SyncStrategy.syncToMe http://127.0.0.1:46509/_mvo/hr/collection1/ has no replicas
[junit4:junit4] 2> 729672 T1925 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46509/_mvo/hr/collection1/
[junit4:junit4] 2> 729672 T1925 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 729940 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 729945 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 729976 T1925 oasc.ZkController.register We are http://127.0.0.1:46509/_mvo/hr/collection1/ and leader is http://127.0.0.1:46509/_mvo/hr/collection1/
[junit4:junit4] 2> 729977 T1925 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46509/_mvo/hr
[junit4:junit4] 2> 729977 T1925 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 729977 T1925 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 729977 T1925 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 729979 T1925 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 729980 T1900 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 729980 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 729981 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 729986 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 729987 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 729988 T1929 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7e5cc2 name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 729988 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 729989 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 729991 T1900 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 730088 T1900 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 730090 T1900 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51911
[junit4:junit4] 2> 730090 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 730091 T1900 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 730091 T1900 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967
[junit4:junit4] 2> 730092 T1900 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/solr.xml
[junit4:junit4] 2> 730092 T1900 oasc.CoreContainer.<init> New CoreContainer 460663623
[junit4:junit4] 2> 730093 T1900 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/'
[junit4:junit4] 2> 730093 T1900 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/'
[junit4:junit4] 2> 730160 T1900 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 730160 T1900 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 730160 T1900 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 730161 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 730161 T1900 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 730161 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 730162 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 730162 T1900 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 730163 T1900 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 730163 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 730167 T1900 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 730167 T1900 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58777/solr
[junit4:junit4] 2> 730168 T1900 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 730168 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 730169 T1940 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ba3b93a name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 730170 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 730171 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 730174 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 730175 T1942 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ba4b692 name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 730176 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 730178 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 731180 T1900 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51911__mvo%2Fhr
[junit4:junit4] 2> 731181 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51911__mvo%2Fhr
[junit4:junit4] 2> 731184 T1922 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 731184 T1929 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 731184 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 731184 T1942 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 731189 T1943 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 731189 T1943 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 731449 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 731449 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:46509__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46509/_mvo/hr"}
[junit4:junit4] 2> 731452 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51911__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51911/_mvo/hr"}
[junit4:junit4] 2> 731453 T1923 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 731453 T1923 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 731456 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 731456 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 731456 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 732190 T1943 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/collection1
[junit4:junit4] 2> 732190 T1943 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 732191 T1943 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 732191 T1943 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 732192 T1943 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/collection1/'
[junit4:junit4] 2> 732193 T1943 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/collection1/lib/README' to classloader
[junit4:junit4] 2> 732193 T1943 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 732228 T1943 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 732277 T1943 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 732278 T1943 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 732284 T1943 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 732817 T1943 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 732824 T1943 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 732828 T1943 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 732838 T1943 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 732841 T1943 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 732844 T1943 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 732845 T1943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 732845 T1943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 732846 T1943 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 732846 T1943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 732846 T1943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 732847 T1943 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 732847 T1943 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371686736967/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/
[junit4:junit4] 2> 732847 T1943 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ae7c8a9
[junit4:junit4] 2> 732848 T1943 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 732848 T1943 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1
[junit4:junit4] 2> 732848 T1943 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index/
[junit4:junit4] 2> 732849 T1943 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 732849 T1943 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index
[junit4:junit4] 2> 732852 T1943 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 732852 T1943 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 732853 T1943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 732854 T1943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 732854 T1943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 732855 T1943 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 732855 T1943 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 732855 T1943 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 732856 T1943 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 732856 T1943 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 732857 T1943 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 732859 T1943 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 732862 T1943 oass.SolrIndexSearcher.<init> Opening Searcher@1b7f3689 main
[junit4:junit4] 2> 732863 T1943 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 732863 T1943 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 732868 T1944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b7f3689 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 732870 T1943 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 732870 T1943 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51911/_mvo/hr collection:collection1 shard:shard1
[junit4:junit4] 2> 732871 T1943 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 732877 T1943 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 732879 T1943 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 732880 T1943 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 732880 T1943 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 732880 T1943 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 732880 T1943 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/ has no replicas
[junit4:junit4] 2> 732880 T1943 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 732881 T1943 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 732961 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 732966 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 732966 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 732966 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 732985 T1943 oasc.ZkController.register We are http://127.0.0.1:51911/_mvo/hr/collection1/ and leader is http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 732985 T1943 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51911/_mvo/hr
[junit4:junit4] 2> 732985 T1943 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 732985 T1943 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 732986 T1943 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 732987 T1943 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 732988 T1900 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 732988 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 732988 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 733092 T1900 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 733094 T1900 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57160
[junit4:junit4] 2> 733094 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 733095 T1900 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 733095 T1900 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968
[junit4:junit4] 2> 733096 T1900 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/solr.xml
[junit4:junit4] 2> 733096 T1900 oasc.CoreContainer.<init> New CoreContainer 460432963
[junit4:junit4] 2> 733097 T1900 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/'
[junit4:junit4] 2> 733097 T1900 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/'
[junit4:junit4] 2> 733172 T1900 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 733173 T1900 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 733173 T1900 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 733174 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 733174 T1900 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 733174 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 733175 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 733175 T1900 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 733176 T1900 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 733176 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 733180 T1900 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 733181 T1900 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58777/solr
[junit4:junit4] 2> 733181 T1900 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 733182 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733183 T1956 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b96a4f1 name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733184 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733185 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 733189 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733189 T1958 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9d7c67 name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733190 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733192 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 734195 T1900 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57160__mvo%2Fhr
[junit4:junit4] 2> 734196 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57160__mvo%2Fhr
[junit4:junit4] 2> 734198 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 734198 T1922 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 734198 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 734199 T1942 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 734199 T1958 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 734199 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 734200 T1929 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 734204 T1959 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 734204 T1959 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 734471 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 734472 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51911__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51911/_mvo/hr"}
[junit4:junit4] 2> 734474 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57160__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57160/_mvo/hr"}
[junit4:junit4] 2> 734475 T1923 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 734475 T1923 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 734477 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 734477 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 734477 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 734477 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 735205 T1959 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/collection1
[junit4:junit4] 2> 735205 T1959 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 735206 T1959 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 735206 T1959 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 735207 T1959 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/collection1/'
[junit4:junit4] 2> 735208 T1959 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/collection1/lib/README' to classloader
[junit4:junit4] 2> 735208 T1959 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 735243 T1959 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 735299 T1959 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 735300 T1959 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 735305 T1959 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 735806 T1959 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 735813 T1959 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 735817 T1959 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 735827 T1959 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 735830 T1959 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 735833 T1959 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 735834 T1959 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 735834 T1959 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 735835 T1959 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 735835 T1959 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 735836 T1959 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 735836 T1959 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 735836 T1959 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371686739968/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/
[junit4:junit4] 2> 735836 T1959 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ae7c8a9
[junit4:junit4] 2> 735837 T1959 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 735837 T1959 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2
[junit4:junit4] 2> 735837 T1959 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index/
[junit4:junit4] 2> 735838 T1959 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 735838 T1959 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index
[junit4:junit4] 2> 735841 T1959 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 735842 T1959 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 735843 T1959 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 735843 T1959 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 735844 T1959 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 735844 T1959 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 735845 T1959 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 735845 T1959 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 735845 T1959 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 735846 T1959 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 735846 T1959 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 735848 T1959 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 735851 T1959 oass.SolrIndexSearcher.<init> Opening Searcher@1bb1986d main
[junit4:junit4] 2> 735852 T1959 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 735852 T1959 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 735862 T1960 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bb1986d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 735864 T1959 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 735864 T1959 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57160/_mvo/hr collection:collection1 shard:shard1
[junit4:junit4] 2> 735866 T1959 oasc.ZkController.register We are http://127.0.0.1:57160/_mvo/hr/collection1/ and leader is http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 735867 T1959 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57160/_mvo/hr
[junit4:junit4] 2> 735867 T1959 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 735867 T1959 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1068 name=collection1 org.apache.solr.core.SolrCore@1bb479d2 url=http://127.0.0.1:57160/_mvo/hr/collection1 node=127.0.0.1:57160__mvo%2Fhr C1068_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57160__mvo%2Fhr, base_url=http://127.0.0.1:57160/_mvo/hr}
[junit4:junit4] 2> 735867 T1961 C1068 P57160 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 735868 T1959 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735868 T1961 C1068 P57160 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 735869 T1961 C1068 P57160 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 735869 T1900 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 735869 T1961 C1068 P57160 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 735869 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 735870 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 735871 T1961 C1068 P57160 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 735876 T1937 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 735948 T1900 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 735950 T1900 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37425
[junit4:junit4] 2> 735951 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 735951 T1900 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 735952 T1900 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850
[junit4:junit4] 2> 735952 T1900 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/solr.xml
[junit4:junit4] 2> 735952 T1900 oasc.CoreContainer.<init> New CoreContainer 453571478
[junit4:junit4] 2> 735953 T1900 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/'
[junit4:junit4] 2> 735953 T1900 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/'
[junit4:junit4] 2> 735982 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735983 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57160__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57160/_mvo/hr"}
[junit4:junit4] 2> 735986 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 735986 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 735986 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 735986 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 736016 T1900 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 736016 T1900 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 736016 T1900 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 736017 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 736017 T1900 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 736017 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 736018 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 736018 T1900 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 736018 T1900 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 736019 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 736022 T1900 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 736023 T1900 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58777/solr
[junit4:junit4] 2> 736023 T1900 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 736024 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 736025 T1973 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b42073f name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 736025 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 736026 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 736030 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 736031 T1975 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b42d32b name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 736031 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 736037 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 736877 T1937 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 736877 T1937 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:57160__mvo%252Fhr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 737039 T1900 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37425__mvo%2Fhr
[junit4:junit4] 2> 737040 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37425__mvo%2Fhr
[junit4:junit4] 2> 737042 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 737042 T1958 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 737042 T1922 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 737043 T1942 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 737043 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737043 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737043 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737043 T1975 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 737044 T1929 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 737050 T1976 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 737051 T1976 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 737489 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 737490 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37425__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37425/_mvo/hr"}
[junit4:junit4] 2> 737490 T1923 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 737490 T1923 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 737495 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737495 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737495 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737495 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 737495 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 738052 T1976 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/collection1
[junit4:junit4] 2> 738052 T1976 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 738053 T1976 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 738053 T1976 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 738054 T1976 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/collection1/'
[junit4:junit4] 2> 738054 T1976 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/collection1/lib/README' to classloader
[junit4:junit4] 2> 738055 T1976 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 738089 T1976 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 738138 T1976 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 738139 T1976 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 738145 T1976 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 738653 T1976 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 738661 T1976 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 738665 T1976 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 738674 T1976 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 738678 T1976 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 738681 T1976 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 738682 T1976 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 738682 T1976 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 738682 T1976 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 738684 T1976 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 738684 T1976 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 738685 T1976 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 738685 T1976 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371686742850/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/
[junit4:junit4] 2> 738685 T1976 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ae7c8a9
[junit4:junit4] 2> 738686 T1976 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 738686 T1976 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3
[junit4:junit4] 2> 738686 T1976 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index/
[junit4:junit4] 2> 738687 T1976 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 738687 T1976 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index
[junit4:junit4] 2> 738690 T1976 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 738691 T1976 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 738692 T1976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 738692 T1976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 738693 T1976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 738693 T1976 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 738694 T1976 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 738694 T1976 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 738694 T1976 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 738695 T1976 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 738695 T1976 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 738698 T1976 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 738701 T1976 oass.SolrIndexSearcher.<init> Opening Searcher@1b2bba68 main
[junit4:junit4] 2> 738702 T1976 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 738702 T1976 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 738707 T1977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b2bba68 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 738709 T1976 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 738709 T1976 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37425/_mvo/hr collection:collection1 shard:shard1
[junit4:junit4] 2> 738711 T1976 oasc.ZkController.register We are http://127.0.0.1:37425/_mvo/hr/collection1/ and leader is http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 738712 T1976 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37425/_mvo/hr
[junit4:junit4] 2> 738712 T1976 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 738712 T1976 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1069 name=collection1 org.apache.solr.core.SolrCore@1b27aca9 url=http://127.0.0.1:37425/_mvo/hr/collection1 node=127.0.0.1:37425__mvo%2Fhr C1069_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:37425__mvo%2Fhr, base_url=http://127.0.0.1:37425/_mvo/hr}
[junit4:junit4] 2> 738712 T1978 C1069 P37425 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 738713 T1976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 738713 T1978 C1069 P37425 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 738713 T1978 C1069 P37425 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 738713 T1978 C1069 P37425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 738714 T1978 C1069 P37425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 738714 T1900 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 738715 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 738716 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 738720 T1935 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 738795 T1900 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 738796 T1900 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36168
[junit4:junit4] 2> 738797 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 738797 T1900 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 738797 T1900 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696
[junit4:junit4] 2> 738798 T1900 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/solr.xml
[junit4:junit4] 2> 738798 T1900 oasc.CoreContainer.<init> New CoreContainer 456903873
[junit4:junit4] 2> 738798 T1900 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/'
[junit4:junit4] 2> 738799 T1900 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/'
[junit4:junit4] 2> 738859 T1900 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 738859 T1900 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 738859 T1900 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 738860 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 738860 T1900 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 738860 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 738861 T1900 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 738861 T1900 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 738861 T1900 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 738862 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 738865 T1900 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 738866 T1900 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58777/solr
[junit4:junit4] 2> 738866 T1900 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 738867 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 738868 T1990 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b54b56a name:ZooKeeperConnection Watcher:127.0.0.1:58777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 738868 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 738869 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 738873 T1900 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 738874 T1992 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b55a5b0 name:ZooKeeperConnection Watcher:127.0.0.1:58777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 738874 T1900 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 738876 T1900 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1070 name=collection1 org.apache.solr.core.SolrCore@1bb479d2 url=http://127.0.0.1:57160/_mvo/hr/collection1 node=127.0.0.1:57160__mvo%2Fhr C1070_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57160__mvo%2Fhr, base_url=http://127.0.0.1:57160/_mvo/hr}
[junit4:junit4] 2> 738878 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51911/_mvo/hr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 738878 T1961 C1070 P57160 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57160/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 738879 T1961 C1070 P57160 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 738879 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 738879 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 738880 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 738880 T1961 C1070 P57160 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 738880 T1961 C1070 P57160 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51911/_mvo/hr/collection1/. core=collection1
[junit4:junit4] 2> 738880 T1961 C1070 P57160 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1071 name=collection1 org.apache.solr.core.SolrCore@1b7388f2 url=http://127.0.0.1:51911/_mvo/hr/collection1 node=127.0.0.1:51911__mvo%2Fhr C1071_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51911__mvo%2Fhr, base_url=http://127.0.0.1:51911/_mvo/hr, leader=true}
[junit4:junit4] 2> 738885 T1936 C1071 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 738885 T1937 C1071 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 738887 T1937 C1071 P51911 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 738887 T1937 C1071 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 738896 T1937 C1071 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 738897 T1937 C1071 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 738897 T1937 C1071 P51911 oass.SolrIndexSearcher.<init> Opening Searcher@1b7b0e98 realtime
[junit4:junit4] 2> 738898 T1937 C1071 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 738898 T1937 C1071 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 738899 T1961 C1070 P57160 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 738899 T1961 C1070 P57160 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 738900 T1938 C1071 P51911 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 738901 T1938 C1071 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 738901 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 738901 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 738902 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 738903 T1937 C1071 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 738904 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 738904 T1961 C1070 P57160 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880
[junit4:junit4] 2> 738904 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b65518b fullCopy=false
[junit4:junit4] 2> 738907 T1936 C1071 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 738907 T1961 C1070 P57160 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 738917 T1961 C1070 P57160 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 738917 T1961 C1070 P57160 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 738919 T1961 C1070 P57160 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 738919 T1961 C1070 P57160 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 738919 T1961 C1070 P57160 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 738920 T1961 C1070 P57160 oass.SolrIndexSearcher.<init> Opening Searcher@1b7ea13c main
[junit4:junit4] 2> 738921 T1960 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b7ea13c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 738922 T1961 C1070 P57160 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880;done=true>>]
[junit4:junit4] 2> 738922 T1961 C1070 P57160 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880
[junit4:junit4] 2> 738923 T1961 C1070 P57160 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index.20130619190545880
[junit4:junit4] 2> 738923 T1961 C1070 P57160 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 738923 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 738924 T1961 C1070 P57160 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 738924 T1961 C1070 P57160 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 738926 T1961 C1070 P57160 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 738999 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 739000 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37425__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37425/_mvo/hr"}
[junit4:junit4] 2> 739002 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57160__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57160/_mvo/hr"}
[junit4:junit4] 2> 739026 T1992 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739026 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739026 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739026 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739026 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739026 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739721 T1935 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 739721 T1935 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:37425__mvo%252Fhr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 739880 T1900 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36168__mvo%2Fhr
[junit4:junit4] 2> 739880 T1900 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36168__mvo%2Fhr
[junit4:junit4] 2> 739882 T1992 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739882 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 739882 T1975 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739883 T1942 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739883 T1958 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739883 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739883 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739883 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739883 T1922 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739885 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 739885 T1929 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739885 T1992 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 739892 T1995 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 739892 T1995 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 740533 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 740534 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36168__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36168/_mvo/hr"}
[junit4:junit4] 2> 740534 T1923 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 740534 T1923 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 740539 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740539 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740539 T1992 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740539 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740539 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740539 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 740893 T1995 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/collection1
[junit4:junit4] 2> 740893 T1995 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 740894 T1995 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 740894 T1995 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 740895 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/collection1/'
[junit4:junit4] 2> 740895 T1995 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/collection1/lib/README' to classloader
[junit4:junit4] 2> 740896 T1995 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 740933 T1995 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 740987 T1995 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 740988 T1995 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 740994 T1995 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 741515 T1995 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 741523 T1995 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 741527 T1995 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 741536 T1995 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741540 T1995 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741543 T1995 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741544 T1995 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741544 T1995 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741544 T1995 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741545 T1995 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741545 T1995 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741545 T1995 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 741546 T1995 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371686745696/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/
[junit4:junit4] 2> 741546 T1995 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ae7c8a9
[junit4:junit4] 2> 741546 T1995 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 741547 T1995 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4
[junit4:junit4] 2> 741547 T1995 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index/
[junit4:junit4] 2> 741547 T1995 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 741547 T1995 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index
[junit4:junit4] 2> 741550 T1995 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 741550 T1995 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 741552 T1995 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 741552 T1995 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 741552 T1995 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 741553 T1995 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 741554 T1995 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 741554 T1995 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 741554 T1995 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 741555 T1995 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 741555 T1995 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 741557 T1995 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 741560 T1995 oass.SolrIndexSearcher.<init> Opening Searcher@1bc146bb main
[junit4:junit4] 2> 741561 T1995 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 741561 T1995 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 741566 T1996 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bc146bb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 741568 T1995 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 741568 T1995 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36168/_mvo/hr collection:collection1 shard:shard1
[junit4:junit4] 2> 741570 T1995 oasc.ZkController.register We are http://127.0.0.1:36168/_mvo/hr/collection1/ and leader is http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 741571 T1995 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36168/_mvo/hr
[junit4:junit4] 2> 741571 T1995 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 741571 T1995 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1072 name=collection1 org.apache.solr.core.SolrCore@1bc4686d url=http://127.0.0.1:36168/_mvo/hr/collection1 node=127.0.0.1:36168__mvo%2Fhr C1072_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:36168__mvo%2Fhr, base_url=http://127.0.0.1:36168/_mvo/hr}
[junit4:junit4] 2> 741571 T1997 C1072 P36168 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 741572 T1995 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741572 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 741573 T1997 C1072 P36168 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 741573 T1900 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 741573 T1997 C1072 P36168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 741573 T1900 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 741574 T1997 C1072 P36168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 741574 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 741578 T1936 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 741579 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741586 T1900 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 741587 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 741588 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1073 name=collection1 org.apache.solr.core.SolrCore@1b27aca9 url=http://127.0.0.1:37425/_mvo/hr/collection1 node=127.0.0.1:37425__mvo%2Fhr C1073_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:37425__mvo%2Fhr, base_url=http://127.0.0.1:37425/_mvo/hr}
[junit4:junit4] 2> 741722 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51911/_mvo/hr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 741722 T1978 C1073 P37425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37425/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 741723 T1978 C1073 P37425 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 741723 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 741723 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 741724 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 741724 T1978 C1073 P37425 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 741724 T1978 C1073 P37425 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51911/_mvo/hr/collection1/. core=collection1
[junit4:junit4] 2> 741724 T1978 C1073 P37425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1074 name=collection1 org.apache.solr.core.SolrCore@1b7388f2 url=http://127.0.0.1:51911/_mvo/hr/collection1 node=127.0.0.1:51911__mvo%2Fhr C1074_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51911__mvo%2Fhr, base_url=http://127.0.0.1:51911/_mvo/hr, leader=true}
[junit4:junit4] 2> 741724 T1938 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 741729 T1937 C1074 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 741736 T1937 C1074 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 741737 T1937 C1074 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 741738 T1937 C1074 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 741738 T1937 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 741739 T1978 C1073 P37425 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 741739 T1978 C1073 P37425 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 741741 T1938 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 741741 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 741742 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 741742 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 741744 T1937 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 741744 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 741745 T1978 C1073 P37425 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721
[junit4:junit4] 2> 741745 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bba4eba fullCopy=false
[junit4:junit4] 2> 741747 T1935 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 741748 T1978 C1073 P37425 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 741750 T1978 C1073 P37425 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 741750 T1978 C1073 P37425 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 741751 T1978 C1073 P37425 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 741751 T1978 C1073 P37425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 741752 T1978 C1073 P37425 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 741752 T1978 C1073 P37425 oass.SolrIndexSearcher.<init> Opening Searcher@1bf97291 main
[junit4:junit4] 2> 741754 T1977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bf97291 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 741754 T1978 C1073 P37425 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721;done=true>>]
[junit4:junit4] 2> 741755 T1978 C1073 P37425 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721
[junit4:junit4] 2> 741755 T1978 C1073 P37425 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index.20130619190548721
[junit4:junit4] 2> 741755 T1978 C1073 P37425 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 741756 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 741756 T1978 C1073 P37425 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 741756 T1978 C1073 P37425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 741757 T1978 C1073 P37425 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 742043 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 742044 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36168__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36168/_mvo/hr"}
[junit4:junit4] 2> 742047 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37425__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37425/_mvo/hr"}
[junit4:junit4] 2> 742066 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742066 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742066 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742066 T1992 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742067 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742067 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 742579 T1936 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 742579 T1936 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:36168__mvo%252Fhr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 742589 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 743591 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1072_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:36168__mvo%2Fhr, base_url=http://127.0.0.1:36168/_mvo/hr}
[junit4:junit4] 2> 744580 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51911/_mvo/hr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 744580 T1997 C1072 P36168 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36168/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 744580 T1997 C1072 P36168 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51911/_mvo/hr/collection1/. core=collection1
[junit4:junit4] 2> 744581 T1997 C1072 P36168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 744582 T1936 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 744586 T1935 C1074 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 744592 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 744596 T1935 C1074 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 744597 T1935 C1074 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 744597 T1935 C1074 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 744597 T1935 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 744598 T1997 C1072 P36168 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 744598 T1997 C1072 P36168 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 744600 T1936 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 744600 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 744600 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 744601 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 744602 T1937 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0
[junit4:junit4] 2> 744603 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 744603 T1997 C1072 P36168 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579
[junit4:junit4] 2> 744603 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd01ec9 fullCopy=false
[junit4:junit4] 2> 744605 T1935 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 744606 T1997 C1072 P36168 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 744609 T1997 C1072 P36168 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 744609 T1997 C1072 P36168 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 744610 T1997 C1072 P36168 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 744611 T1997 C1072 P36168 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 744611 T1997 C1072 P36168 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 744611 T1997 C1072 P36168 oass.SolrIndexSearcher.<init> Opening Searcher@1bc79fa7 main
[junit4:junit4] 2> 744613 T1996 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bc79fa7 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 744614 T1997 C1072 P36168 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579;done=true>>]
[junit4:junit4] 2> 744614 T1997 C1072 P36168 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579
[junit4:junit4] 2> 744614 T1997 C1072 P36168 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index.20130619190551579
[junit4:junit4] 2> 744615 T1997 C1072 P36168 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 744615 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 744615 T1997 C1072 P36168 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 744615 T1997 C1072 P36168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 744616 T1997 C1072 P36168 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 745074 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 745075 T1923 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36168__mvo%2Fhr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36168/_mvo/hr"}
[junit4:junit4] 2> 745078 T1958 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745078 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745078 T1992 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745078 T1929 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745078 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745078 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 745594 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 745595 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1075 name=collection1 org.apache.solr.core.SolrCore@1b7c09b9 url=http://127.0.0.1:46509/_mvo/hr/collection1 node=127.0.0.1:46509__mvo%2Fhr C1075_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:46509__mvo%2Fhr, base_url=http://127.0.0.1:46509/_mvo/hr, leader=true}
[junit4:junit4] 2> 745604 T1915 C1075 P46509 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 745607 T1915 C1075 P46509 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 745608 T1915 C1075 P46509 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 745636 T1915 C1075 P46509 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 745637 T1915 C1075 P46509 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 745638 T1915 C1075 P46509 oass.SolrIndexSearcher.<init> Opening Searcher@1bf450d0 main
[junit4:junit4] 2> 745638 T1915 C1075 P46509 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 745639 T1926 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bf450d0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 745639 T1915 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4] 2> ASYNC NEW_CORE C1076 name=collection1 org.apache.solr.core.SolrCore@1bb479d2 url=http://127.0.0.1:57160/_mvo/hr/collection1 node=127.0.0.1:57160__mvo%2Fhr C1076_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57160__mvo%2Fhr, base_url=http://127.0.0.1:57160/_mvo/hr}
[junit4:junit4] 2> 745651 T1954 C1076 P57160 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51911/_mvo/hr/collection1/, StdNode: http://127.0.0.1:57160/_mvo/hr/collection1/, StdNode: http://127.0.0.1:37425/_mvo/hr/collection1/, StdNode: http://127.0.0.1:36168/_mvo/hr/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 745653 T1951 C1076 P57160 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 745653 T1936 C1074 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1072_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:36168__mvo%2Fhr, base_url=http://127.0.0.1:36168/_mvo/hr}
[junit4:junit4] 2> 745658 T1986 C1072 P36168 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1073_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37425__mvo%2Fhr, base_url=http://127.0.0.1:37425/_mvo/hr}
[junit4:junit4] 2> 745660 T1968 C1073 P37425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 745676 T1951 C1076 P57160 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 745677 T1951 C1076 P57160 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 745678 T1951 C1076 P57160 oass.SolrIndexSearcher.<init> Opening Searcher@1bfeff6e main
[junit4:junit4] 2> 745678 T1951 C1076 P57160 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 745679 T1960 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bfeff6e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 745680 T1951 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 27
[junit4:junit4] 2> 745707 T1936 C1074 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 745707 T1968 C1073 P37425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 745707 T1936 C1074 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 745707 T1986 C1072 P36168 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 745708 T1936 C1074 P51911 oass.SolrIndexSearcher.<init> Opening Searcher@1bf5c0bb main
[junit4:junit4] 2> 745709 T1986 C1072 P36168 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 745708 T1968 C1073 P37425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 745709 T1936 C1074 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 745710 T1944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bf5c0bb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 745710 T1986 C1072 P36168 oass.SolrIndexSearcher.<init> Opening Searcher@1bbea242 main
[junit4:junit4] 2> 745711 T1936 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 745711 T1968 C1073 P37425 oass.SolrIndexSearcher.<init> Opening Searcher@1c052d91 main
[junit4:junit4] 2> 745712 T1968 C1073 P37425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 745712 T1996 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bbea242 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 745712 T1977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c052d91 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 745713 T1986 C1072 P36168 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 745713 T1968 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 53
[junit4:junit4] 2> 745713 T1986 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> 745714 T1954 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 745715 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 745717 T1937 C1074 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 745718 T1952 C1076 P57160 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 745719 T1970 C1073 P37425 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 745720 T1987 C1072 P36168 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 747723 T1918 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438317810494210048)} 0 1
[junit4:junit4] 2> 747728 T1987 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={update.distrib=FROMLEADER&_version_=-1438317810498404352&update.from=http://127.0.0.1:51911/_mvo/hr/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438317810498404352)} 0 1
[junit4:junit4] 2> 747729 T1970 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={update.distrib=FROMLEADER&_version_=-1438317810498404352&update.from=http://127.0.0.1:51911/_mvo/hr/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438317810498404352)} 0 2
[junit4:junit4] 2> 747729 T1953 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={update.distrib=FROMLEADER&_version_=-1438317810498404352&update.from=http://127.0.0.1:51911/_mvo/hr/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438317810498404352)} 0 1
[junit4:junit4] 2> 747730 T1936 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438317810498404352)} 0 4
[junit4:junit4] 2> 747730 T1969 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 5
[junit4:junit4] 2> 747735 T1916 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[0 (1438317810505744384)]} 0 2
[junit4:junit4] 2> 747742 T1988 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438317810510987264)]} 0 1
[junit4:junit4] 2> 747743 T1968 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438317810510987264)]} 0 2
[junit4:junit4] 2> 747743 T1951 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438317810510987264)]} 0 1
[junit4:junit4] 2> 747744 T1937 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:37425/_mvo/hr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438317810510987264)]} 0 6
[junit4:junit4] 2> 747744 T1971 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[0]} 0 7
[junit4:junit4] 2> 747747 T1917 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[1 (1438317810520424448)]} 0 1
[junit4:junit4] 2> 747753 T1985 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438317810524618752)]} 0 0
[junit4:junit4] 2> 747754 T1970 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438317810524618752)]} 0 1
[junit4:junit4] 2> 747755 T1952 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438317810524618752)]} 0 1
[junit4:junit4] 2> 747755 T1935 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:57160/_mvo/hr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438317810524618752)]} 0 4
[junit4:junit4] 2> 747756 T1954 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[1]} 0 7
[junit4:junit4] 2> 747758 T1915 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[2 (1438317810531958784)]} 0 0
[junit4:junit4] 2> 747762 T1938 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:57160/_mvo/hr/collection1/]
[junit4:junit4] 2> 747762 T1938 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:57160/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:true
[junit4:junit4] 2> 747763 T1938 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:37425/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:false
[junit4:junit4] 2> 747763 T1938 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:36168/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:false
[junit4:junit4] 2> 747766 T1969 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[2 (1438317810537201664)]} 0 1
[junit4:junit4] 2> 747766 T1986 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[2 (1438317810537201664)]} 0 1
[junit4:junit4] 2> 747767 T1938 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:57160/_mvo/hr/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[2 (1438317810537201664)]} 0 5
[junit4:junit4] 2> 747767 T1953 C1076 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[2]} 0 7
[junit4:junit4] 2> 747770 T1918 C1075 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={wt=javabin&version=2} {add=[3 (1438317810544541696)]} 0 0
[junit4:junit4] 2> 747773 T1936 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:57160/_mvo/hr/collection1/]
[junit4:junit4] 2> 747773 T1936 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:57160/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:true
[junit4:junit4] 2> 747773 T1936 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:37425/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:false
[junit4:junit4] 2> 747773 T1936 C1074 P51911 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:36168/_mvo/hr/collection1/ against:[http://127.0.0.1:57160/_mvo/hr/collection1/] result:false
[junit4:junit4] 2> 747776 T1968 C1073 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[3 (1438317810548736000)]} 0 0
[junit4:junit4] 2> 747776 T1988 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:51911/_mvo/hr/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[3 (1438317810548736000)]} 0 0
[junit4:junit4] 2> 747777 T1936 C1074 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={distrib.from=http://127.0.0.1:36168/_mvo/hr/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&wt=javabin&version=2} {add=[3 (1438317810548736000)]} 0 5
[junit4:junit4] 2> 747778 T1987 C1072 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={test.distrib.skip.servers=http://127.0.0.1:57160/_mvo/hr/collection1/&test.distrib.skip.servers=http://127.0.0.1:37425/_mvo/hr/collection1/&wt=javabin&version=2} {add=[3]} 0 6
[junit4:junit4] 2> 747780 T1916 C1075 P46509 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C1077 name=collection1 org.apache.solr.core.SolrCore@1b7c09b9 url=http://127.0.0.1:46509/_mvo/hr/collection1 node=127.0.0.1:46509__mvo%2Fhr C1077_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:46509__mvo%2Fhr, base_url=http://127.0.0.1:46509/_mvo/hr, leader=true}
[junit4:junit4] 2> 747836 T1916 C1077 P46509 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 747837 T1916 C1077 P46509 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 747839 T1916 C1077 P46509 oass.SolrIndexSearcher.<init> Opening Searcher@1ae3019b main
[junit4:junit4] 2> 747840 T1916 C1077 P46509 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 747841 T1926 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ae3019b main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 747841 T1916 C1077 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 61
[junit4:junit4] 2> ASYNC NEW_CORE C1078 name=collection1 org.apache.solr.core.SolrCore@1b27aca9 url=http://127.0.0.1:37425/_mvo/hr/collection1 node=127.0.0.1:37425__mvo%2Fhr C1078_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37425__mvo%2Fhr, base_url=http://127.0.0.1:37425/_mvo/hr}
[junit4:junit4] 2> 747843 T1971 C1078 P37425 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51911/_mvo/hr/collection1/, StdNode: http://127.0.0.1:57160/_mvo/hr/collection1/, StdNode: http://127.0.0.1:37425/_mvo/hr/collection1/, StdNode: http://127.0.0.1:36168/_mvo/hr/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C1079 name=collection1 org.apache.solr.core.SolrCore@1bb479d2 url=http://127.0.0.1:57160/_mvo/hr/collection1 node=127.0.0.1:57160__mvo%2Fhr C1079_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57160__mvo%2Fhr, base_url=http://127.0.0.1:57160/_mvo/hr}
[junit4:junit4] 2> 747845 T1951 C1079 P57160 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C1080 name=collection1 org.apache.solr.core.SolrCore@1bc4686d url=http://127.0.0.1:36168/_mvo/hr/collection1 node=127.0.0.1:36168__mvo%2Fhr C1080_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:36168__mvo%2Fhr, base_url=http://127.0.0.1:36168/_mvo/hr}
[junit4:junit4] 2> 747845 T1985 C1080 P36168 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C1081 name=collection1 org.apache.solr.core.SolrCore@1b7388f2 url=http://127.0.0.1:51911/_mvo/hr/collection1 node=127.0.0.1:51911__mvo%2Fhr C1081_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51911__mvo%2Fhr, base_url=http://127.0.0.1:51911/_mvo/hr, leader=true}
[junit4:junit4] 2> 747845 T1937 C1081 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 747845 T1970 C1078 P37425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 747897 T1951 C1079 P57160 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 747897 T1985 C1080 P36168 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 747897 T1951 C1079 P57160 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 747898 T1985 C1080 P36168 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 747900 T1951 C1079 P57160 oass.SolrIndexSearcher.<init> Opening Searcher@1b0f96ca main
[junit4:junit4] 2> 747900 T1985 C1080 P36168 oass.SolrIndexSearcher.<init> Opening Searcher@1b0ebc96 main
[junit4:junit4] 2> 747900 T1951 C1079 P57160 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 747901 T1996 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:299)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1847)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:139)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:909)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2>
[junit4:junit4] 2> 747901 T1985 C1080 P36168 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 747901 T1960 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b0f96ca main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):c2)}
[junit4:junit4] 2> 747902 T1996 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b0ebc96 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 747903 T1951 C1079 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 747903 T1985 C1080 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 747907 T1937 C1081 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 747908 T1937 C1081 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 747911 T1937 C1081 P51911 oass.SolrIndexSearcher.<init> Opening Searcher@1b1c28c3 main
[junit4:junit4] 2> 747912 T1937 C1081 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 747913 T1944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b1c28c3 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 747914 T1937 C1081 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 69
[junit4:junit4] 2> 747947 T1970 C1078 P37425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 747947 T1970 C1078 P37425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 747950 T1970 C1078 P37425 oass.SolrIndexSearcher.<init> Opening Searcher@1b147998 main
[junit4:junit4] 2> 747950 T1970 C1078 P37425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 747951 T1977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b147998 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 747952 T1970 C1078 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 107
[junit4:junit4] 2> 747952 T1971 C1078 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 109
[junit4:junit4] 2> 747953 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 747954 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 747955 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 747957 T1935 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 747959 T1952 C1079 P57160 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 747961 T1969 C1078 P37425 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 747963 T1986 C1080 P36168 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 747964 T1900 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 747969 T1968 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 747969 T1968 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 747977 T1938 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 747977 T1938 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 747978 T1938 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr START replicas=[http://127.0.0.1:57160/_mvo/hr/collection1/, http://127.0.0.1:37425/_mvo/hr/collection1/, http://127.0.0.1:36168/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 747979 T1970 C1078 P37425 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747979 T1988 C1080 P36168 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747979 T1954 C1079 P57160 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747979 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Received 5 versions from 127.0.0.1:37425/_mvo/hr/collection1/
[junit4:junit4] 2> 747980 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Our versions are newer. ourLowThreshold=1438317810498404352 otherHigh=1438317810537201664
[junit4:junit4] 2> 747981 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Received 5 versions from 127.0.0.1:36168/_mvo/hr/collection1/
[junit4:junit4] 2> 747981 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Our versions are newer. ourLowThreshold=1438317810498404352 otherHigh=1438317810537201664
[junit4:junit4] 2> 747981 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Received 3 versions from 127.0.0.1:57160/_mvo/hr/collection1/
[junit4:junit4] 2> 747982 T1938 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr Our versions are newer. ourLowThreshold=1438317810498404352 otherHigh=1438317810524618752
[junit4:junit4] 2> 747982 T1938 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51911/_mvo/hr DONE. sync succeeded
[junit4:junit4] 2> 747983 T1938 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 747983 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: try and ask http://127.0.0.1:57160/_mvo/hr/collection1/ to sync
[junit4:junit4] 2> 747983 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: try and ask http://127.0.0.1:37425/_mvo/hr/collection1/ to sync
[junit4:junit4] 2> 747984 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: try and ask http://127.0.0.1:36168/_mvo/hr/collection1/ to sync
[junit4:junit4] 2> 747985 T1971 C1078 P37425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37425/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 747984 T1953 C1079 P57160 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57160/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 747985 T1987 C1080 P36168 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36168/_mvo/hr START replicas=[http://127.0.0.1:51911/_mvo/hr/collection1/] nUpdates=100
[junit4:junit4] 2> 747986 T1936 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747987 T1935 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747987 T1953 C1079 P57160 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57160/_mvo/hr Received 5 versions from 127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 747987 T1987 C1080 P36168 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:36168/_mvo/hr Received 5 versions from 127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 747988 T1987 C1080 P36168 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:36168/_mvo/hr Our versions are newer. ourLowThreshold=1438317810498404352 otherHigh=1438317810537201664
[junit4:junit4] 2> 747988 T1953 C1079 P57160 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:57160/_mvo/hr Requesting updates from 127.0.0.1:51911/_mvo/hr/collection1/n=2 versions=[1438317810548736000, 1438317810537201664]
[junit4:junit4] 2> 747989 T1937 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 747988 T1987 C1080 P36168 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36168/_mvo/hr DONE. sync succeeded
[junit4:junit4] 2> 747989 T1971 C1078 P37425 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37425/_mvo/hr Received 5 versions from 127.0.0.1:51911/_mvo/hr/collection1/
[junit4:junit4] 2> 747990 T1987 C1080 P36168 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={sync=http://127.0.0.1:51911/_mvo/hr/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 747990 T1971 C1078 P37425 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:37425/_mvo/hr Our versions are newer. ourLowThreshold=1438317810498404352 otherHigh=1438317810537201664
[junit4:junit4] 2> 747991 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: sync completed with http://127.0.0.1:36168/_mvo/hr/collection1/
[junit4:junit4] 2> 747991 T1971 C1078 P37425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37425/_mvo/hr DONE. sync succeeded
[junit4:junit4] 2> 747992 T1971 C1078 P37425 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={sync=http://127.0.0.1:51911/_mvo/hr/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=8
[junit4:junit4] 2> 747993 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: sync completed with http://127.0.0.1:37425/_mvo/hr/collection1/
[junit4:junit4] 2> 747993 T1936 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={distrib=false&getUpdates=1438317810548736000,1438317810537201664&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 748000 T1953 C1079 P57160 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438317810537201664), 3 (1438317810548736000)]} 0 2
[junit4:junit4] 2> 748000 T1953 C1079 P57160 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57160/_mvo/hr DONE. sync succeeded
[junit4:junit4] 2> 748001 T1953 C1079 P57160 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/get params={sync=http://127.0.0.1:51911/_mvo/hr/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=17
[junit4:junit4] 2> 748001 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:51911/_mvo/hr/collection1/: sync completed with http://127.0.0.1:57160/_mvo/hr/collection1/
[junit4:junit4] 2> 748002 T1938 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=26
[junit4:junit4] 2> 748002 T1968 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=34
[junit4:junit4] 2> 748003 T1900 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 748003 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 748004 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 748004 T1900 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 748006 T1917 C1077 P46509 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 748027 T1917 C1077 P46509 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 748027 T1917 C1077 P46509 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 748029 T1917 C1077 P46509 oass.SolrIndexSearcher.<init> Opening Searcher@1b25ab66 main
[junit4:junit4] 2> 748030 T1917 C1077 P46509 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 748031 T1926 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b25ab66 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 748031 T1917 C1077 P46509 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 748033 T1985 C1080 P36168 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51911/_mvo/hr/collection1/, StdNode: http://127.0.0.1:57160/_mvo/hr/collection1/, StdNode: http://127.0.0.1:37425/_mvo/hr/collection1/, StdNode: http://127.0.0.1:36168/_mvo/hr/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 748035 T1969 C1078 P37425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 748035 T1988 C1080 P36168 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 748035 T1952 C1079 P57160 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 748036 T1935 C1081 P51911 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 748056 T1969 C1078 P37425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 748057 T1969 C1078 P37425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 748058 T1969 C1078 P37425 oass.SolrIndexSearcher.<init> Opening Searcher@1b0f3aea main
[junit4:junit4] 2> 748059 T1969 C1078 P37425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 748060 T1977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b0f3aea main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 748061 T1969 C1078 P37425 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 748077 T1935 C1081 P51911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 748077 T1988 C1080 P36168 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty4/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 748077 T1935 C1081 P51911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 748078 T1988 C1080 P36168 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 748080 T1935 C1081 P51911 oass.SolrIndexSearcher.<init> Opening Searcher@1af0e240 main
[junit4:junit4] 2> 748080 T1935 C1081 P51911 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 748081 T1988 C1080 P36168 oass.SolrIndexSearcher.<init> Opening Searcher@1ae47128 main
[junit4:junit4] 2> 748081 T1988 C1080 P36168 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 748081 T1944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1af0e240 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 748082 T1996 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ae47128 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 748082 T1935 C1081 P51911 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 46
[junit4:junit4] 2> 748083 T1988 C1080 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 748107 T1952 C1079 P57160 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371686733401/jetty2/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 748108 T1952 C1079 P57160 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 748112 T1952 C1079 P57160 oass.SolrIndexSearcher.<init> Opening Searcher@1b2c82de main
[junit4:junit4] 2> 748112 T1952 C1079 P57160 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 748113 T1960 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b2c82de main{StandardDirectoryReader(segments_5:5:nrt _0(4.4):c2 _1(4.4):c2)}
[junit4:junit4] 2> 748114 T1952 C1079 P57160 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 79
[junit4:junit4] 2> 748114 T1985 C1080 P36168 oasup.LogUpdateProcessor.finish [collection1] webapp=/_mvo/hr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 81
[junit4:junit4] 2> 748115 T1900 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 748118 T1937 C1081 P51911 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 748120 T1954 C1079 P57160 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 748122 T1970 C1078 P37425 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 748123 T1987 C1080 P36168 oasc.SolrCore.execute [collection1] webapp=/_mvo/hr path=/select params={tests=checkShar
[...truncated too long message...]
licas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:46509__mvo%2Fhr",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46509/_mvo/hr",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:46509__mvo%2Fhr (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:51911__mvo%2Fhr (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:36168__mvo%2Fhr (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37425__mvo%2Fhr (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894861760299018-4-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894861760299022-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89894861760299020-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:36168__mvo%2Fhr",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:36168/_mvo/hr"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89894861760299011-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:46509__mvo%2Fhr",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46509/_mvo/hr"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89894861760299018-127.0.0.1:37425__mvo%2Fhr-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894861760299022-127.0.0.1:51911__mvo%2Fhr-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894861760299011-127.0.0.1:46509__mvo%2Fhr-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89894861760299020-127.0.0.1:36168__mvo%2Fhr-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89894861760299011-127.0.0.1:46509__mvo%2Fhr-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=1106578B97D21F1D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sr_CS -Dtests.timezone=Jamaica -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 100s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51911/_mvo/hr/collection1lastClient and got 251 from http://127.0.0.1:37425/_mvo/hr/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([1106578B97D21F1D:90E0D993E08D7F21]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 826852 T1900 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 100457 T1899 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 826918 T1975 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 826918 T1975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 826918 T1975 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=MockRandom), text=MockVariableIntBlock(baseBlockSize=109), _version_=PostingsFormat(name=MockRandom), rnd_b=MockFixedIntBlock(blockSize=1754), intDefault=PostingsFormat(name=MockRandom), id=MockFixedIntBlock(blockSize=1754), timestamp=PostingsFormat(name=MockRandom), a_t=PostingsFormat(name=MockRandom), range_facet_sl=MockVariableIntBlock(baseBlockSize=109), range_facet_si=PostingsFormat(name=Direct), other_tl1=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=Direct), a_si=PostingsFormat(name=Direct)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=sr_CS, timezone=Jamaica
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Oracle Corporation 1.6.0_37 (32-bit)/cpus=8,threads=1,free=149569152,total=327081984
[junit4:junit4] 2> NOTE: All tests run in this JVM: [LukeRequestHandlerTest, TestPerFieldSimilarity, ShowFileRequestHandlerTest, TestSchemaVersionResource, TestSerializedLuceneMatchVersion, TestDocSet, SoftAutoCommitTest, SliceStateTest, TestReversedWildcardFilterFactory, BasicDistributedZkTest, TestPluginEnable, TestGroupingSearch, TestCloudManagedSchema, TestConfig, IndexSchemaTest, FileBasedSpellCheckerTest, PluginInfoTest, QueryEqualityTest, CacheHeaderTest, BadComponentTest, OverseerCollectionProcessorTest, TestTrie, TestSolrDeletionPolicy2, DirectSolrConnectionTest, ResourceLoaderTest, TestQueryUtils, FieldMutatingUpdateProcessorTest, ResponseLogComponentTest, TestValueSourceCache, TestDynamicFieldResource, AutoCommitTest, SchemaVersionSpecificBehaviorTest, TestJoin, TestCharFilters, UpdateRequestProcessorFactoryTest, DistributedTermsComponentTest, TestLFUCache, TestDocumentBuilder, TestStressRecovery, AliasIntegrationTest, MBeansHandlerTest, TestPropInjectDefaults, LoggingHandlerTest, BadCopyFieldTest, CoreAdminHandlerTest, BinaryUpdateRequestHandlerTest, TestXIncludeConfig, DocValuesTest, TestFastWriter, CSVRequestHandlerTest, TestManagedSchema, TestSweetSpotSimilarityFactory, UnloadDistributedZkTest, SynonymTokenizerTest, CachingDirectoryFactoryTest, RegexBoostProcessorTest, JsonLoaderTest, TestPseudoReturnFields, TestCloudManagedSchemaAddField, WordBreakSolrSpellCheckerTest, QueryElevationComponentTest, TermVectorComponentTest, AlternateDirectoryTest, QueryResultKeyTest, TestWriterPerf, SpatialFilterTest, MinimalSchemaTest, TestSchemaResource, TestSolrDeletionPolicy1, DateFieldTest, TestSchemaSimilarityResource, TestSolrCoreProperties, UUIDFieldTest, CollectionsAPIDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 100.81s, 1 test, 1 failure <<< FAILURES!
[...truncated 595 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1255 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 67 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jrockit-jdk1.6.0_37-R28.2.5-4.1.0 -XnoOpt
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure